Wrangle Data with Python in Azure Machine Learning

You should already be logged in to the Azure portal. If you have problems, be sure to use an InPrivate or Incognito window in your browser, and be sure to use the login credentials provided with the lab.

1. After you have logged in, from the resource group already deployed for you, select the Azure Machine Learning workspace, also already deployed for you.

2. From the workspace, launch Azure Machine Learning studio; a separate tab will open.

3. Once in the studio, choose Notebooks, and add a new file to open a new notebook. Name the file anything you like, and leave the file type defaulted.

4. At the top, next to Compute, confirm that the compute instance deployed for you is running. Start it, if necessary, but do not create a new instance or try to use the serverless Spark option. If you do, the lab will fail.

5. When prompted, authenticate/validate your connection to the compute instance.

6. Add a line of sample code to the first notebook cell:

   print("Hello World")
   

7. Confirm the kernel is running by clicking the Run button to the left of the cell to execute the cell. The run should complete in less than a few seconds.

Suggestions Before You Start

• Create a new Code cell in your notebook for each coding task in order to wrangle iteratively and interactively.

• Name your DataFrame my_dataframe to more easily check your work and understand the code hints.

• Enter the code, below, at the end of each cell to return all the rows in the dataset. This will enable you to study the data and any changes you've made to it.

  my_dataframe.head(1000)
  

• Using any method to modify one or more rows in the data, pass in the inplace=True parameter to make the change in memory in the dataset, without needing to return any data. For example:

  my_dataframe.someMethod(aParameter, anotherParameter, inplace=True)
  

Coding Tasks

1. Import the pandas library and read the contents of a csv file into a DataFrame. The csv file is located on GitHub, accessible at a URL provided in the Additional Resources section of this lab. View the DataFrame contents.
2. For some of the rows in the dataset, two status columns have None or NaN, instead of values, indicating empty, or NULL columns in those rows. Replace the empty values with the word "Unknown."
3. Assume that you have cleaned up missing values as much as you can. Delete any rows in the dataset that still have any empty columns.
4. You should note three exact duplicate rows in the data. Keep the first occurrence of the duplicate set, and delete the other two.
5. BONUS TASK: The de-duping process will cause the rows to land out of original order, which makes it harder to spot the remaining row from the set of duplicates. Execute a method that restores the order based on the index (the sequence number on the far left of each row).
6. Save the DataFrame to a new csv file located in the same folder as the notebook you created at the beginning of the lab.

Coding Hints

Spoiler alert: These hints might provide you with more help than you need. If they provide you with less help than you need, then it is probably time to visit the lab guide or solution video. Coding hint numbers refer to the coding task numbers.

1. The import statement is import pandas as pd, and the DataFrame method is read_csv.
2. The DataFrame method is fillna.
3. The DataFrame method is dropna.
4. The DataFrame method is drop_duplicates
5. The bonus re-sorting DataFrame method is sort_index.
6. The method to persist data is to_csv.