{"cells":[{"cell_type":"markdown","id":"348c957c","metadata":{},"source":["\n","\n","# 目次\n","\n","1. 機械学習の例\n","- ここではシンプルなLinear Regressionを例に出します\n","\n","2. アニーリングに移行してみる\n","- 重みをバイナリ変数で扱う方法を紹介します"]},{"cell_type":"markdown","id":"2f63fff7","metadata":{},"source":["# 機械学習の例 | Linear Regression\n","\n","Linear Regression(線型回帰)は確率を回帰する機械学習モデルです。$x$が入力、$Y$を出力とし、パラメータ$\\begin{pmatrix}\\beta_0 \\\\ \\vdots \\\\ \\beta_k \\end{pmatrix}$を学習させます。\n","\n","線型回帰のモデルは以下のような式で定義されます。\n","\n","$$\n","Y = \\beta_0 x_0 + \\cdots + \\beta_k x_k = \\sum_{i=0}^{k} \\beta_{i} x_i\n","$$"]},{"cell_type":"code","execution_count":1,"id":"4572010b","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:53.484072Z","iopub.status.busy":"2021-10-28T13:08:53.483902Z","iopub.status.idle":"2021-10-28T13:08:53.938349Z","shell.execute_reply":"2021-10-28T13:08:53.937930Z","shell.execute_reply.started":"2021-10-28T13:08:53.484030Z"},"tags":[]},"outputs":[],"source":["import numpy as np\n","import pandas as pd\n","from sklearn.linear_model import LinearRegression\n","from sklearn.metrics import roc_auc_score, accuracy_score, mean_squared_error"]},{"cell_type":"markdown","id":"b640b4bc","metadata":{},"source":["そしたらLinear Regressionを使って学習を行ってみようと思います。\n","\n","`calc_y`関数で定義されるように、各特徴量に対して1か0の重みを掛け、それらの和をtargetとして与えました。\n","回帰係数が[1, 0, 1, 1]になったら完璧ですね。"]},{"cell_type":"code","execution_count":2,"id":"aa99cd26","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:54.483299Z","iopub.status.busy":"2021-10-28T13:08:54.483137Z","iopub.status.idle":"2021-10-28T13:08:54.495745Z","shell.execute_reply":"2021-10-28T13:08:54.495271Z","shell.execute_reply.started":"2021-10-28T13:08:54.483270Z"},"tags":[]},"outputs":[{"data":{"text/html":["

\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","

	0	1	2	3	target
0	1	1	0	1	2
1	0	1	0	1	1
2	0	1	0	0	0
3	1	0	1	0	2
4	0	1	0	1	1

\n","

"],"text/plain":[" 0 1 2 3 target\n","0 1 1 0 1 2\n","1 0 1 0 1 1\n","2 0 1 0 0 0\n","3 1 0 1 0 2\n","4 0 1 0 1 1"]},"execution_count":2,"metadata":{},"output_type":"execute_result"}],"source":["def calc_y(x):\n"," output = x[0] * 1 + x[1] * 0 + x[2] * 1 + x[3] * 1\n"," # output = 1 if res >= 2 \n"," return output\n","\n","\n","nrows = 100\n","nfea = 4\n","# nrows x nfeaの行列を作る\n","df = np.random.randint(2, size=(nrows, nfea)).astype(bool)\n","target = []\n","\n","for i in range(nrows):\n"," t = int(calc_y(df[i, :]))\n"," target.append(t)\n","\n","\n","df = pd.DataFrame(df).astype(int)\n","df[\"target\"] = target\n","\n","df.head()"]},{"cell_type":"markdown","id":"69b7e9d5","metadata":{},"source":["`sklearn`の`LinearRegression`モジュールで学習を行います。特にパラメータの指定は行いません。"]},{"cell_type":"code","execution_count":3,"id":"03ffb477","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:54.876771Z","iopub.status.busy":"2021-10-28T13:08:54.876145Z","iopub.status.idle":"2021-10-28T13:08:54.884247Z","shell.execute_reply":"2021-10-28T13:08:54.883896Z","shell.execute_reply.started":"2021-10-28T13:08:54.876695Z"},"tags":[]},"outputs":[{"data":{"text/plain":["LinearRegression()"]},"execution_count":3,"metadata":{},"output_type":"execute_result"}],"source":["lr = LinearRegression()\n","lr.fit(df.iloc[:, 0:4], df.iloc[:, 4])"]},{"cell_type":"markdown","id":"e2e1470e","metadata":{},"source":["回帰係数は`calc_y`関数で定義した通りと、意図した結果が得られました。"]},{"cell_type":"code","execution_count":4,"id":"8caaa8a0","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:55.324316Z","iopub.status.busy":"2021-10-28T13:08:55.323685Z","iopub.status.idle":"2021-10-28T13:08:55.335276Z","shell.execute_reply":"2021-10-28T13:08:55.334344Z","shell.execute_reply.started":"2021-10-28T13:08:55.324238Z"},"tags":[]},"outputs":[{"data":{"text/plain":["array([1.00000000e+00, 2.22044605e-16, 1.00000000e+00, 1.00000000e+00])"]},"execution_count":4,"metadata":{},"output_type":"execute_result"}],"source":["# 回帰係数の確認\n","\n","lr.coef_"]},{"cell_type":"markdown","id":"35a2db07","metadata":{},"source":["# アニーリングに移行してみる\n","\n","どうやってやるか、端的に説明すると「重みをバイナリ変数にし、二乗誤差をエネルギーとして定式化する」になります。\n","\n","## 重みをバイナリ変数にする\n","\n","バイナリ変数は0-1の二値をとる変数です。\n","\n","$$\n","\\hat{y} = \\beta_0 x_0 + \\cdots + \\beta_k x_k\n","$$\n","\n","↑これが　↓こう ($q$は$\\{0, 1\\}$を取るバイナリ変数です)\n","\n","$$\n","\\hat{y} = q_0 x_0 + \\cdots + q_k x_k\n","$$\n","\n","線型回帰の重みは任意の実数がとれますが、バイナリ変数を使うことで二値しかとれないことがわかります。\n","\n","### 二値じゃ全然役に立たないのでは？\n","\n","ここで、バイナリ変数にすることで生じる課題を解決する方法を紹介します。しかし、これからの解説では上に書いた0と1の二値の重みを使用するので、興味がある人が読んでください\n","\n","#### 逆量子化 | より多くの整数値をとる\n","\n","例えば下のような式で$q_0$を定義したらどうでしょうか。$q_0$は0から10の11個の整数値をとることができます。\n","\n","$$\n","q_0 = \\sum_{i=0}^9 q_{0, i}\n","$$\n","\n","\n","実数の値を離散値に丸めることを離散値といいますが、これは離散値をたくさん用意することで実数の値に近づける\"逆量子化\"を行っています。ちなみに量子化という言葉は学術的に使用されているワードですが、逆量子化は私が命名した言葉なので、この分野における正しいワードかはわかりません。\n","\n","#### スケーリング | 整数値だけでなく、小数も扱えるように\n","\n","逆量子化では整数値しかとれませんでしたが、各バイナリ変数に対して小数の重み付けをしてあげることで\"全体として\"小数の数を扱うことができます。\n","\n","ex. 1 \n","逆量子化の例に出した0 ~ 10の重みの範囲を0 ~ 1.0にする\n","\n","$$\n","q_0 = \\sum_{i=0}^9 0.1 q_{0, i}\n","$$\n","\n","\n","ex. 2 \n","重み$\\begin{pmatrix}w_0 \\\\ \\vdots \\\\ w_k \\end{pmatrix}$を使う。一般化させただけ。\n","\n","$$\n","q_0 = \\sum_{i=0}^{k} w_i q_{0, i}\n","$$\n","\n"]},{"cell_type":"markdown","id":"8530364e","metadata":{},"source":["## 二乗誤差で定式化する\n","\n","誤差関数を二乗誤差としたらアニーリングにおけるエネルギーと誤差関数を同一視することができます。\n","\n","ターゲット$y$に対して予測値を$\\hat{y}$としたとき、エネルギー(誤差関数)$H$は以下のように与えられます。\n","\n","$$\n","H = (y - \\hat{y}) ^2\n","$$\n","\n","$\\hat{y}$は`重みをバイナリ変数にする`チャプターで定義したので、それを代入します。\n","\n","$$\n","H = (y - (q_0 x_0 + \\cdots + q_k x_k))^2 = (y - \\sum_0^k q_i x_i)^2\n","$$"]},{"cell_type":"markdown","id":"7073699f","metadata":{},"source":["## 実装してみる\n","\n","先程、様々な小数の値をとることのできる逆量子化を紹介しましたが、今回の重みは0か1をとる1つのバイナリ変数のみで実装します。"]},{"cell_type":"code","execution_count":5,"id":"0f7dd45f","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:56.113686Z","iopub.status.busy":"2021-10-28T13:08:56.113038Z","iopub.status.idle":"2021-10-28T13:08:56.141956Z","shell.execute_reply":"2021-10-28T13:08:56.141519Z","shell.execute_reply.started":"2021-10-28T13:08:56.113609Z"},"tags":[]},"outputs":[],"source":["from amplify import IsingPoly, IsingMatrix, gen_symbols, sum_poly, BinaryPoly\n","from amplify import Solver, decode_solution, BinarySymbolGenerator\n","from amplify.client import FixstarsClient\n","import json"]},{"cell_type":"markdown","id":"3985c123","metadata":{"execution":{"iopub.execute_input":"2021-10-28T12:57:39.030997Z","iopub.status.busy":"2021-10-28T12:57:39.030742Z","iopub.status.idle":"2021-10-28T12:57:39.035495Z","shell.execute_reply":"2021-10-28T12:57:39.034471Z","shell.execute_reply.started":"2021-10-28T12:57:39.030966Z"}},"source":["自分はローカルのファイルからトークンを読み込んでいますが、動かすときには`token読み込み`を消して以下ようにベタ書きしてください。\n","\n","余談ですが、ローカルから読み込むようにすればトークンの流出が防げたり、トークンを更新してもコードを変える必要がなくなったりします。\n","\n","```python\n","client.token = \"token\"\n","```"]},{"cell_type":"code","execution_count":6,"id":"7fd980d8","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:56.518183Z","iopub.status.busy":"2021-10-28T13:08:56.517989Z","iopub.status.idle":"2021-10-28T13:08:56.522205Z","shell.execute_reply":"2021-10-28T13:08:56.521545Z","shell.execute_reply.started":"2021-10-28T13:08:56.518161Z"},"tags":[]},"outputs":[],"source":["client = FixstarsClient()\n","\n","\"\"\"token読み込み　はじまり\"\"\"\n","token_path = \"/home/yuma/.amplify/token.json\"\n","\n","with open(token_path) as f:\n"," client.token = json.load(f)[\"AMPLIFY_TOKEN\"]\n","\n","\"\"\"token読み込み　おわり\"\"\"\n"," \n","client.parameters.timeout = 100\n","solver = Solver(client)"]},{"cell_type":"markdown","id":"b9d907e1","metadata":{},"source":["学習させる4つのバイナリ変数(0か1をとるイジング変数)を用意します。"]},{"cell_type":"code","execution_count":7,"id":"f36974d0","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:56.957920Z","iopub.status.busy":"2021-10-28T13:08:56.957259Z","iopub.status.idle":"2021-10-28T13:08:56.962748Z","shell.execute_reply":"2021-10-28T13:08:56.962371Z","shell.execute_reply.started":"2021-10-28T13:08:56.957840Z"},"tags":[]},"outputs":[{"data":{"text/plain":["[q_0, q_1, q_2, q_3]"]},"execution_count":7,"metadata":{},"output_type":"execute_result"}],"source":["gen = BinarySymbolGenerator()\n","q = gen.array(nfea)\n","\n","q"]},{"cell_type":"markdown","id":"c2f0235f","metadata":{},"source":["エネルギーの定式化を行います。各行のデータに対して二乗誤差を計算し、エネルギーに足していきます。"]},{"cell_type":"code","execution_count":8,"id":"44f7c772","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:57.358854Z","iopub.status.busy":"2021-10-28T13:08:57.358716Z","iopub.status.idle":"2021-10-28T13:08:57.379046Z","shell.execute_reply":"2021-10-28T13:08:57.378341Z","shell.execute_reply.started":"2021-10-28T13:08:57.358839Z"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["------------dfの1行目------------\n"," 0 1 2 3 target\n","0 1 1 0 1 2\n","-----------y_hat, y-----------\n","q_0 + q_1 + q_3, 2\n","------------エネルギー-------------\n","2 q_0 q_1 + 2 q_0 q_3 + 2 q_1 q_3 - 3 q_0 - 3 q_1 - 3 q_3 + 4\n"]}],"source":["f = 0\n","\n","for i in range(nrows):\n"," # amplifyのドキュメントでは標準のsum関数は推奨されていません。\n"," # イテレートする回数が多い時はamplifyのsum_poly関数を使いましょう。\n"," \n"," y_hat = sum(\n"," q[j] * df.iloc[i, j]\n"," for j in range(nfea)\n"," )\n"," \n"," # 二乗誤差\n"," f += (df.iloc[i, 4] - y_hat) ** 2\n"," \n"," if i == 0:\n"," print(f\"{'dfの1行目':-^30}\")\n"," print(df.iloc[[0], :])\n"," print(f\"{'y_hat, y':-^30}\")\n"," print(f\"{y_hat}, {df.iloc[0, 4]}\")\n"," print(f\"{'エネルギー':-^30}\")\n"," print(f\"{f}\")"]},{"cell_type":"markdown","id":"2a59e9d4","metadata":{},"source":["定式化は完了したので、Amplify AEに計算式を投げて解きます。"]},{"cell_type":"code","execution_count":9,"id":"ae29d87c","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:58.266554Z","iopub.status.busy":"2021-10-28T13:08:58.266419Z","iopub.status.idle":"2021-10-28T13:08:58.725407Z","shell.execute_reply":"2021-10-28T13:08:58.724804Z","shell.execute_reply.started":"2021-10-28T13:08:58.266539Z"},"tags":[]},"outputs":[{"data":{"text/plain":["array([1., 0., 1., 1.])"]},"execution_count":9,"metadata":{},"output_type":"execute_result"}],"source":["result = solver.solve(f)\n","binary_weight = q.decode(result[0].values)\n","\n","binary_weight"]},{"cell_type":"markdown","id":"8383b8b8","metadata":{},"source":["上の解のエネルギー（誤差関数）はいくらでしょうか！　0です！！　最強のAIができました。\n","\n","これはyを定義した`calc_y`の定義式、Linear Regressionの解とも一致しています。"]},{"cell_type":"code","execution_count":10,"id":"a56406dc","metadata":{"execution":{"iopub.execute_input":"2021-10-28T13:08:58.860080Z","iopub.status.busy":"2021-10-28T13:08:58.859426Z","iopub.status.idle":"2021-10-28T13:08:58.872235Z","shell.execute_reply":"2021-10-28T13:08:58.870187Z","shell.execute_reply.started":"2021-10-28T13:08:58.860000Z"},"tags":[]},"outputs":[{"data":{"text/plain":["0.0"]},"execution_count":10,"metadata":{},"output_type":"execute_result"}],"source":["result[0].energy"]},{"cell_type":"markdown","id":"5d13f2d6","metadata":{},"source":["以上を通してアニーリングを用いて機械学習を行うことができます。\n","\n","今回は限定的な線型回帰でしたが、この考え方を使うことで幅広い機械学習モデルを実装することが可能になります！"]}],"metadata":{"kernelspec":{"display_name":"amplify","language":"python","name":"amplify"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.8.8"}},"nbformat":4,"nbformat_minor":5}

chizuchizu

アニーリングマシンで機械学習