I'm not exactly sure how do I train the data, is it training each image (2883 input channels) for a single neural net, training a batch at a time, or the entire training set?
Dataloaders If you are not sure what dataloaders are, please refer to the tutorial or PyTorch document. With dataloaders, you can iterate through the datasets easily.In this MP, we have provided dataloaders for training set and testing set, respectively. \# Generate dataloaders \# you don't need to call these functions in MP, train_loader and test_loader are passed to your function a train_set_p, test_set_p = reader.Preprocess(train_set, test_set) train_loader, test_loader = reader.Get_DataLoaders(train_set_p, train_labels, test_set_p, test_labels, bat Each iteration below returns a batch of train_features and train_labels (in this MP, we set up batch size equal to 100, so each batch contains 100 feature and label tensors respectively). You can pass the feature batch to your neural network, and then compare the label batch with your predictions. Let's iterate over the dataset and see what each batch looks like. Labels: ship: 0 , automobile: 1 , dog: 2 , frog: 3 , horse: 4 batch_index =0 \# Iterate over the dataloader for features, labels in train_loader: \# you can train your network with the feature and label batches here print("Batch \#", batch_index) print("Feature shape:", features.shape) print (features, " " ) print("Label shape:", labels.shape) print(labels, " " ) batch_index +=1 Batch \# 0 Feature shape: torch.Size ([100,2883]) tensor ([[0.3525,0.3150,0.6970,,1.2380,1.2221,1.3589], [0.8684,0.9081,0.9093,,1.0154,1.0167,0.9519], [0.3932,0.3839,0.2262,,0.6230,0.5951,0.5977], , [0.4756,0.1494,0.5631,,0.6230,0.7689,1.1299], [1.4793,1.5012,1.5324,,0.9199,0.9219,0.8737], [0.5027,0.5816,0.5309,,1.0631,1.0483,0.9989]]) Label shape: torch. Size ([100]) tensor ([0,0,0,3,1,1,0,1,4,4,4,2,1,0,3,4,1,1,3,1,1,4,1,1, 4,4,3,2,0,4,2,1,0,1,0,3,0,1,0,4,4,0,4,0,3,2,2,3, 4,2,3,3,1,2,3,3,2,2,2,2,1,0,4,0,3,3,0,3,1,0,0,2, 1,3,0,0,2,4,3,2,3,2,2,3,3,2,0,3,4,0,1,0,3,1,0,0, 4,2,0,1]) Batch \# 1 Feature shape: torch.Size ([100,2883]) tensor ([[1.7381,1.7541,1.7771,,1.7523,1.7487,1.7403], [0.3127,0.3333,0.3647,,0.2890,0.3107,0.3159], [1.0594,1.0368,1.4863,,1.4024,0.6899,0.7229], , [0.3661,0.4115,0.4108,,0.6018,0.4320,0.4510], [1.7237,1.7633,1.7539,,2.3038,2.2808,2.2511], [0.6249,0.7609,0.3232,,0.4003,0.2898,1.3276]]) This section aims to help you get some ideas of the datasets we are using. Note that in this MP we have provided dataloaders for you, so you to load any datasets by yourself. The dataset consists of 3131 colored (RGB) images (a modified subset of the CIFAR-10 dataset, provided by Alex Krizhevsky). This set is split for you into training examples and development examples. The function Load_dataset) in reader.py will unpack the dataset file (you don't need to call this function in the MP), returning images and labels for the training and development sets. Note that the images have been flattened, therefore the dimension of one image sample is 2883(31313). import reader \# filepath to data filepath = "./data /mp_data" \# Load datasets, you don't need to call this function in the MP train_set, train_labels, test_set, test_labels = reader.Load_dataset(filepath) print("Shape of train set:", train_set.shape) print("Shape of test set:", test_set.shape) Shape of train set: (2813, 2883) Shape of test set: (937,2883)
