我有一個(gè) 3 軸數(shù)據(jù)的 DataFrames，帶有一個(gè)成員資格標(biāo)簽，我用它來分組：df = pd.DataFrame( [[0, 1, 2, 0],
[-1, 0, 1, 0],
[-2, 0, 3, 1],
[1, 1, 3, 1],
[1, 0, 2, 2],
[1, 0, 3, 2],
[6, 2, 1, 5],
[-4, 3, 0, 5],
[1, 0, -1, 6],
[0, 0, 3, 6]], columns = ['x', 'y', 'z', 'member'])我的目標(biāo)有點(diǎn)做作：我希望找到每個(gè)組的點(diǎn)與下一個(gè)組之間的成對(duì)距離，從小到大排序。這就是我所說的交錯(cuò)的意思：n_skipn_skip例如，對(duì)于 ，我希望找到以下距離：n_skip=2帶有 --> against 的行member == 0member == 1, 2帶有 --> 反對(duì)的行member == 1member == 2, 5帶有 --> 反對(duì)的行member == 2member == 5, 6帶有 --> 反對(duì)的行member == 5member == 6沒有計(jì)算 。member == 6有沒有一種高性能的方法可以在沒有嵌套的for循環(huán)的情況下做到這一點(diǎn)？這個(gè)問題的答案中提到了這一點(diǎn)。直觀地說，我無法使用傳統(tǒng)的方法來并行化 Pandas DataFrame 上的函數(shù)。將函數(shù)應(yīng)用于一組交錯(cuò)組的快速方法是什么？applyEDIT1 我的解決方案（僅適用于一個(gè)軸）： ## Heading ### Organize by group membership groups = df.groupby('member') # Define constants max_member = 6 n_skip = 2 start_row = 0 matrix = np.zeros((df.shape[0], df.shape[0])) # Iterate for each group for i in range(max_member): try: pts_curr = groups.get_group(i) except KeyError: continue # Save end row index end_row = start_row + pts_curr.shape[0] # Save start col index start_col = end_row # Grab the destination group nodes for j in range(i+1, int(np.min([i+n_skip+1, max_member]))): try: pts_clr_next = groups.get_group(j) except KeyError: continue # Save end col index end_col = start_col + pts_clr_next.shape[0] # Calculate cdist z_sq = cdist(pts_curr[['z']], pts_next[['z']]) # Save results in matrix at right positions matrix[start_row:end_row, start_col:end_col] = z_sq # update col index start_col = end_col # update row index start_row = end_row
1 回答
慕哥6287543
TA貢獻(xiàn)1831條經(jīng)驗(yàn) 獲得超10個(gè)贊
4K 行的交叉合并還不錯(cuò)（產(chǎn)生大約 16M 行）。讓我們嘗試交叉合并和查詢：
n = 2
# dummy key
df['dummy'] = 1
# this is the member group number
df['rank'] = df['member'].rank(method='dense')
# cross merge and filter
new_df = (df.merge(df, on='dummy')
.query('rank_x<rank_y<=rank_x+@n')
)
# euclidean distance
dist = (new_df[['x_x','y_x','z_x']].sub(new_df[['x_y','y_y','z_y']].values)**2).sum(1)**.5
# output dataframe with member label
pd.DataFrame({'member1':new_df['member_x'], 'member2':new_df['member_y'],
'dist':dist})
輸出：
member1 member2 dist
2 0 1 2.449490
3 0 1 1.414214
4 0 2 1.414214
5 0 2 1.732051
12 0 1 2.236068
13 0 1 3.000000
14 0 2 2.236068
15 0 2 2.828427
24 1 2 3.162278
25 1 2 3.000000
26 1 5 8.485281
27 1 5 4.690416
34 1 2 1.414214
35 1 2 1.000000
36 1 5 5.477226
37 1 5 6.164414
46 2 5 5.477226
47 2 5 6.164414
48 2 6 3.000000
49 2 6 1.414214
56 2 5 5.744563
57 2 5 6.557439
58 2 6 4.000000
59 2 6 1.000000
68 5 6 5.744563
69 5 6 6.633250
78 5 6 5.916080
79 5 6 5.830952
選項(xiàng) 2：如果數(shù)據(jù)幀較大，則循環(huán)可能還不錯(cuò)：
from scipy.spatial.distance import cdist
ret = []
for i in set(df['rank']):
this_group = df['rank']==i
other_groups = df['rank'].between(i,i+n, inclusive=False)
t = df.loc[this_group,['x','y','z']].values
o = df.loc[other_groups,['x','y','z']].values
ret.append(cdist(t,o).ravel())
dist = np.concatenate(ret)
- 1 回答
- 0 關(guān)注
- 157 瀏覽
添加回答
舉報(bào)
0/150
提交
取消