我有兩個 Keras（Tensorflow 后端）模型，它們堆疊在一起形成一個組合模型：small_model 和 In: (None,K), Out: (None,K)large_model 和 In: (None,N,K), Out: (None,1)combined_model( N x small_model -> large_model) 與In: (None,N,K), Out: (None,1)large_model需要N堆疊輸出small_model作為輸入。我可以定義N small_model共享權(quán)重的 s，然后連接它們的輸出（從技術(shù)上講，我需要堆疊它們），然后將其發(fā)送到large_model，如下面的代碼所示。我的問題是我需要能夠為非常大的N( > 10**6)執(zhí)行此操作，并且我當(dāng)前的解決方案在創(chuàng)建模型時使用了大量內(nèi)存和時間，即使對于N ~ 10**2.我希望有一個解決方案可以并行發(fā)送N數(shù)據(jù)點small_model（就像將批次提供給模型時所做的那樣），收集這些點（使用 Keras 歷史，以便反向傳播是可能的）并將其發(fā)送到large_model，而不必定義 的N實例small_model。列出的三個模型的輸入和輸出形狀不應(yīng)改變，但當(dāng)然可以定義其他中間模型。當(dāng)前不滿意的解決方案（假設(shè)small_model和large_model已經(jīng)存在，并且N,K已定義）：from keras.layers import Input, Lambdafrom keras.models import Modelfrom keras import backend as Kdef build_small_model_on_batch(): def distribute_inputs_to_small_model(input): return [small_model(input[:,i]) for i in range(N)] def stacker(list_of_tensors): return K.stack(list_of_tensors, axis=1) input = Input(shape=(N,K,)) small_model_outputs = Lambda(distribute_inputs_to_small_model)(input) stacked_small_model_outputs = Lambda(stacker)(small_model_outputs) return Model(input, stacked_small_model_outputs)def build_combined(): input = Input(shape=(N,K,)) stacked_small_model_outputs = small_model_on_batch(input) output = large_model(stacked_small_model_outputs) return Model(input, output)small_model_on_batch = build_small_model_on_batch()combined = build_combined()
1 回答
四季花海
TA貢獻(xiàn)1811條經(jīng)驗 獲得超5個贊
您可以使用TimeDistributed層包裝器來做到這一點：
from keras.layers import Input, Dense, TimeDistributed
from keras.models import Sequential, Model
N = None # Use fixed value if you do not want variable input size
K = 20
def small_model():
inputs = Input(shape=(K,))
# Define the small model
# Here it is just a single dense layer
outputs = Dense(K, activation='relu')(inputs)
return Model(inputs=inputs, outputs=outputs)
def large_model():
inputs = Input(shape=(N, K))
# Define the large model
# Just a single neuron here
outputs = Dense(1, activation='relu')(inputs)
return Model(inputs=inputs, outputs=outputs)
def combined_model():
inputs = Input(shape=(N, K))
# The TimeDistributed layer applies the given model
# to every input across dimension 1 (N)
small_model_out = TimeDistributed(small_model())(inputs)
# Apply large model
outputs = large_model()(small_model_out)
return Model(inputs=inputs, outputs=outputs)
model = combined_model()
model.compile(loss='mean_squared_error', optimizer='sgd')
model.summary()
輸出：
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, None, 20) 0
_________________________________________________________________
time_distributed_1 (TimeDist (None, None, 20) 420
_________________________________________________________________
model_2 (Model) (None, None, 1) 21
=================================================================
Total params: 441
Trainable params: 441
Non-trainable params: 0
_________________________________________________________________
添加回答
舉報
0/150
提交
取消