imdb导mysql_keras如何导入本地下载的imdb数据集？

第一步 下载数据集到本地

提取码：9h3u

存储位置：C:/用户/用户名/.keras/datasets

(用户名不同人不一样，可能电脑不一样存储位置也略有差异)

第二步 导入数据集

import keras

import numpy as np

# load data

from keras.datasets import imdb

(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)

----------

查看数据集是否导入正确

print(train_labels[0]) #1

print(max([max(sequence) for sequence in train_data])) #9999

----------

遇到的一些小问题以及解决办法：

若出现了几个问题，最后差不多是这样：raise ValueError("Object arrays cannot be loaded when " ValueError: Object arrays cannot be loaded ……

这说明numpy版本太高了，我一开始的版本是1.16.4，之后转换成了1.16.2

版本转换：

cmd输入xxxxxxxxxxxxxxxx numpy==1.16.2

xxxxxxxxxx为https://mirrors.tuna.tsinghua.edu.cn/help/pypi/中代码，可加快下载速度，直接复制，只需要将some-package改成numpy==1.16.2即可

第三步 电影评论二分类完整代码示例

import keras

import numpy as np

# load data

from keras.datasets import imdb

(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)

print(train_labels[0]) #1

print(max([max(sequence) for sequence in train_data])) #9999

# 将索引解码为单词，需要下载imdb_word_index.json至C:/用户/用户名/.keras/datasets

# 链接：https://pan.baidu.com/s/1kkmpXrr1tkFtg7D3LX_lcw 提取码：wzjw

word_index = imdb.get_word_index() #将单词映射为整数索引的字典

reverse_word_index = dict([(value, key) for (key, value) in word_index.items()]) #键值颠倒，将整数索引映射为单词

decoded_review = ' '.join([reverse_word_index.get(i - 3, '?') for i in train_data[0]])

#print(decoded_review)

# 对列表进行one-shot编码，eg.将[3,5]转换成[0.0,0.0,1.0,0.0,1.0,0.0,0.0,0.0,...]

def vectorize_sequences(sequences, dimension=10000):

results = np.zeros((len(sequences), dimension))

for i, sequence in enumerate(sequences):

results[i,sequence] = 1. #将 results[i] 的指定索引设为 1

return results

# handle input data

x_train = vectorize_sequences(train_data)

x_test = vectorize_sequences(test_data)

#print(x_train[0]) #[0. 1. 1. ... 0. 0. 0.]

# handle output data

y_train = np.asarray(train_labels).astype('float32')

y_test = np.asarray(test_labels).astype('float32')

# 验证集预留

x_val = x_train[:10000]

partial_x_train = x_train[10000:]

y_val = y_train[:10000]

partial_y_train = y_train[10000:]

# build model

from keras import models

from keras import layers

model = models.Sequential()

model.add(layers.Dense(16, activation='relu', input_shape=(10000,)))

model.add(layers.Dense(16, activation='relu'))

model.add(layers.Dense(1, activation='sigmoid'))

# train model

model.compile(optimizer='rmsprop',

loss='binary_crossentropy',

metrics=['accuracy'])

history = model.fit(partial_x_train,

partial_y_train,

epochs=20,

batch_size=512,

validation_data=(x_val, y_val))

history_dict = history.history

#print(history_dict.keys()) #dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])

# 绘制训练损失、验证损失、训练精度、验证精度

import matplotlib.pyplot as plt

# plot loss

acc = history.history['acc']

val_acc = history.history['val_acc']

loss = history.history['loss']

val_loss = history.history['val_loss']

epochs = range(1, len(acc)+1)

plt.plot(epochs, loss, 'bo', label='Training loss') #blue o

plt.plot(epochs, val_loss, 'b', label='Validation loss') #blue solid line

plt.title('Training and validation loss')

plt.xlabel('Epochs')

plt.ylabel('Loss')

plt.legend()

plt.show()

# plot accuracy

plt.clf() #清空图像

plt.plot(epochs, acc, 'bo', label='Training acc')

plt.plot(epochs, val_acc, 'b', label='Validation acc')

plt.title('Training and validation accuracy')

plt.xlabel('Epochs')

plt.ylabel('Accuracy')

plt.legend()

plt.show()

----------

此处验证集用来确定训练NN可采用的最佳epoch，训练集--->NN参数，得出epochs=4

用新参数搭建的NN去训练train_data，注释掉history以及history之后的代码：

model.fit(x_train, y_train, epochs=4, batch_size=512)

results = model.evaluate(x_test, y_test)

print(results)

print(model.predict(x_test))

------------

进一步实验的实验结果，(控制变量：

[0.29455984374523164, 0.88312] #原结构共三层

[0.2833905682277679, 0.88576] #共两层

[0.30949291754722597, 0.87984] #神经元个数为32

[0.08610797638118267, 0.88308] #mse

[0.32080167996406556, 0.87764] #用tanh代替relu

选定的结构较为合适。

总结

以上是生活随笔为你收集整理的imdb导mysql_keras如何导入本地下载的imdb数据集？的全部内容，希望文章能够帮你解决所遇到的问题。

如果觉得生活随笔网站内容还不错，欢迎将生活随笔推荐给好友。