工作循環(huán)，預(yù)期結(jié)果我正在嘗試使用非常大的數(shù)據(jù)集對(duì)代碼中的慢速 for 循環(huán)進(jìn)行矢量化，以根據(jù)測(cè)試刪除重復(fù)項(xiàng)。結(jié)果應(yīng)該只保留前 3 個(gè)元素唯一的元素，而第 4 個(gè)元素是所有重復(fù)項(xiàng)中最大的元素。例如in = np.array(((0, 12, 13, 1), (0, 12, 13, 10), (1, 12, 13, 2)))應(yīng)該成為out = np.array(((0, 12, 13, 10), (1, 12, 13, 2)))使用 for 循環(huán)實(shí)現(xiàn)這一點(diǎn)很簡單，但正如我提到的，它非常慢。unique = np.unique(in[:, :3], axis=0)out = np.empty((0, 4))for i in unique: out = np.vstack((out, np.hstack((i[:], np.max(in[np.all(in[:, :3] == i[:], axis=1)][:, 3])))))我試過的 (1)當(dāng)我嘗試通過將每個(gè)替換為以下索引來刪除帶有索引的 for 循環(huán)i[:]時(shí)unique[np.arange(unique.shape[0])]：out = np.vstack((out, np.hstack((unique[np.arange(unique.shape[0])], np.max(in[np.all(in[:, :3].astype(int) == unique[np.arange(unique.shape[0])], axis=1)][:, 3])))))Numpy 抱怨輸入形狀連同所有：Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<__array_function__ internals>", line 6, in all File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 2351, in all return _wrapreduction(a, np.logical_and, 'all', axis, None, out, keepdims=keepdims) File "/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py", line 90, in _wrapreduction return ufunc.reduce(obj, axis, dtype, out, **passkwargs)numpy.AxisError: axis 1 is out of bounds for array of dimension 0我試過的（2）根據(jù)輸入此問題時(shí) StackOverflow 的建議（Broadcasting/Vectorizing inner and outer for loops in python/NumPy）：newout = np.vstack((newout, np.hstack((tempunique[:, None], np.max(inout[np.all(inout[:, :3].astype(int) == tempunique[:, None], axis=1)][:, 3])))))我收到一個(gè)錯(cuò)誤，抱怨輸入和輸出之間的大小不匹配：Traceback (most recent call last): File "<stdin>", line 1, in <module>IndexError: boolean index did not match indexed array along dimension 0; dimension is 3 but corresponding boolean dimension is 2重述問題是否有正確的方法來廣播我的索引以消除 for 循環(huán)？
1 回答
HUH函數(shù)
TA貢獻(xiàn)1836條經(jīng)驗(yàn) 獲得超4個(gè)贊
我對(duì)您的用例了解不夠，無法確定是否值得引入 Pandas，但在 Pandas 中有效地這樣做只需要幾行代碼：
import numpy as np
import pandas as pd
in_array = np.array(((0, 12, 13, 1), (0, 12, 13, 10), (1, 12, 13, 2)))
in_df = pd.DataFrame(in_array)
# group by unique combinations of the 0th, 1st, and 2nd columns, then take the
# max of the 3rd column in each group. `reset_index` change cols 0-2 from index
# back to normal columns
out_df = in_df.groupby([0, 1, 2])[3].max().reset_index()
out_array = out_df.values
print(out_array)
# Output:
# [[ 0 12 13 10]
# [ 1 12 13 2]]
一個(gè)簡單的計(jì)時(shí)測(cè)試表明，使用 Pandas 處理一個(gè) 100000 行隨機(jī)生成的輸入數(shù)組需要 0.0117 秒，而使用 for 循環(huán)實(shí)現(xiàn)需要 2.6103 秒。
添加回答
舉報(bào)
0/150
提交
取消