我正在使用 Twitter 的情緒數(shù)據(jù)集對(duì)情緒進(jìn)行分類。為了實(shí)現(xiàn)這一點(diǎn)，我寫了下面的代碼，但是當(dāng)我訓(xùn)練它時(shí)，我得到了損失 NaN。我無(wú)法理解問(wèn)題所在。雖然我設(shè)法找到了問(wèn)題的解決方案，但為什么問(wèn)題首先發(fā)生在我不明白的地方。代碼 ：import pandas as pdimport numpy as npimport recols = ["id","text","emotion","intensity"]anger_df_train = pd.read_csv("D:/Dataset/twitter_emotion/train/anger.csv",delimiter='\t',names=cols)fear_df_train = pd.read_csv("D:/Dataset/twitter_emotion/train/fear.csv",delimiter='\t',names=cols)joy_df_train = pd.read_csv("D:/Dataset/twitter_emotion/train/joy.csv",delimiter='\t',names=cols)sadness_df_train = pd.read_csv("D:/Dataset/twitter_emotion/train/sadness.csv",delimiter='\t',names=cols)df_train = pd.concat([anger_df_train,fear_df_train,joy_df_train,sadness_df_train])import spacynlp = spacy.load('en_core_web_md')doc = nlp("The big grey dog ate all of the chocolate, but fortunately he wasn't sick!")def spacy_tokenizer(sentence): emails = '[A-Za-z0-9]+@[a-zA-Z].[a-zA-Z]+' websites = '(http[s]*:[/][/])[a-zA-Z0-9]' mentions = '@[A-Za-z0-9]+' sentence = re.sub(emails,'',sentence) sentence = re.sub(websites,'',sentence) sentence = re.sub(mentions,'',sentence) sentence_list=[word.lemma_ for word in nlp(sentence) if not (word.is_stop or word.is_space or word.like_num or len(word)==1)] return ' '.join(sentence_list)import tensorflow as tffrom tensorflow.keras.preprocessing.text import Tokenizerfrom tensorflow.keras.preprocessing.sequence import pad_sequencesdf_train['new_text']=df_train['text'].apply(spacy_tokenizer)tokenizer = Tokenizer(num_words=10000)tokenizer.fit_on_texts(df_train['new_text'].values)sequences = tokenizer.texts_to_sequences(df_train['new_text'].values)text_embedding = np.zeros((len(word_index)+1,300))for word,i in word_index.items(): text_embedding[i]=nlp(word).vectorlabels = df_train['emotion'].unique()label_tokenizer = Tokenizer()label_tokenizer.fit_on_texts(labels)
1 回答
蝴蝶刀刀
TA貢獻(xiàn)1801條經(jīng)驗(yàn) 獲得超8個(gè)贊
sparse_categorical_crossentropy導(dǎo)致的原因NaN是我們?cè)?code>Tokenize the Labels使用的時(shí)候Tokenizer，Train Labels Array生成的，如下圖：
array([[1], [2], [3], [4], [1], [2], [3]])
但是，如果sparse_categorical_crossentropy必須應(yīng)用 Loss，Train Labels Array則應(yīng)如下所示：
array([0, 1, 2, 3, 0, 1, 2])
因此，我們可以sparse_categorical_crossentropy使用以下代碼使您的代碼與 Loss 一起工作：
label_map={'anger': 0, 'fear': 1, 'joy': 2, 'sadness': 3}
df['Labels'] = df['Labels'].map(label_map)
sparse_categorical_labels = df['Labels'].values
X_train,X_test,y_train,y_test = train_test_split(train_padd,sparse_categorical_labels,test_size=0.2,shuffle=True)添加回答
舉報(bào)
0/150
提交
取消