This is the second blog entry in this sequence that delves into data wrangling. This entry is also part the data-science portfolio project.
To view the previous post in this blog series, click below:
O*NET Data Wrangling
Observations
The data in the O*NET database did not warrant as much wrangling, at least at this stage. I was able to pull the data from PostgreSQL into Jupyter using SQLAlchemy, sort through it, and come to an initial understanding without too much fuss.
There area few things that I have not yet attempted to finish. Some of the rows contain null values and these may need to be dropped. I don’t want to go through with that yet, until I know for sure.
The BLS Wage dataset and the O*NET datasets are supposed to be joinable on the OCC Codes. These are codes that the US Government creates to categorize the various types of common occupations.
A problem here can be seen in that the O*NET database has a slightly different format for the OCC Codes, and I do not yet know why. The first several digits are all the same, but then the O*NET database has a .XX additional suffix, the meaning of which is unclear.
When I go to join the databases, I will need to resolve this issue.
My full Jupyter Notebook where I wrangle the O*NET dataset is below.
After the notebook, the blog post briefly continues.
If that notebook does not load for any reason, take a look on Github for the file here:
Link to Github O*NET Training Dataset Wrangle
Conclusion
Both the O*NET and BLS datasets are ready for further exploration. I’ll be revisiting the wrangling stage as needed as I continue.
Artificial Intelligence Transparency Report
No artificial intelligence was used for the writing of this blog post.
In the above story, I’ve endeavored to explain the times when I turned to AI for support.
In the Github directory linked below, there is a public diary where I document some of my process and track my occasional AI usage.
How You Can Help
I need your help to become established as a teacher and storyteller.
Here is a link to a blog post that describes how a supportive reader can help me in my quest.
In short, you can…
Buy a copy of my children’s novella, Westly: A Spider’s Tale
Like, comment and share