LinkedIn Professional - Generating Tech Jargon with DeepAI Text Generator
Table of Contents
- LinkedIn Professional - Project Overview
- LinkedIn Professional - Generating Tech Jargon with DeepAI Text Generator
- LinkedIn Professional - Using LinkedIn's API
- LinkedIn Professional - Deploying a Serverless project
LinkedIn Professional - Generating Tech Jargon with DeepAI Text Generator
In this post we're going to start looking into how we would be able to create AI generated text sentences using some simple and free online services. This includes the code required to pull down random skills from the ThoughtWorks technology radar website as well.
I've updated the architecture diagram below to include the components that we will be touching for this portion of the guide.
Prerequisites
To deploy this project yourself you will require a couple things to be pre-setup:
- AWS CLI (with credentials)
- t04glovern/linkedin-professional Repository: Clone down the repository I've created from https://github.com/t04glovern/linkedin-professional
ThoughtWorks technology radar skill scraping
Let's get started writing some basic python code for scraping the skills from the ThoughtWorks technology radar website. To start with let's take a look at the website and work out what we need to do to extract the data.
We can see that there are 4 pages for each of the sections and each are available at a different subpage of the website.
I wrote the following code immediately that we will be able to use shortly.
radar_base_url = 'https://www.thoughtworks.com/radar/'
quadrants = ['techniques', 'tools', 'platforms', 'languages-and-frameworks']
Each of the subpages have 4 sections; adopt, trial, assess and hold. So I we're going to install a library called Beautiful Soup that can handle website parsing programmatically. This library can be installed with the following
pip install beautifulsoup4==4.9.3 requests==2.25.1
Then I wrote the following code to parse out the skills from HTML, and then a function that leverages that other function to select a random skill
from bs4 import BeautifulSoup
import os
radar_base_url = 'https://www.thoughtworks.com/radar/'
quadrants = ['techniques', 'tools', 'platforms', 'languages-and-frameworks']
skill_items = {}
def skills_from_html(html_target_radar):
skill_group = {}
options = ['adopt', 'trial', 'assess', 'hold']
for option in options:
radar_option = html_target_radar.find(id=option)
items = radar_option.find_all('li')
skill_group[option] = list(map(lambda x: x.find('a').text, items))
return skill_group
def get_skill_items():
# Only update the list if empty
if not skill_items:
for quadrant in quadrants:
target_radar_url = radar_base_url + quadrant
target_radar_page = requests.get(target_radar_url)
target_radar_soup = BeautifulSoup(
target_radar_page.content, 'html.parser')
target_radar = target_radar_soup.find(id='responsive-tech-radar')
skill_items[quadrant] = skills_from_html(target_radar)
return skill_items
Awesome! we have a way to pull out skill lists now. I've also included a nice little way to generate a sentence in a structure that we can submit to a sentence generating.
def random_skill(skill_items):
quadrants = ['techniques', 'tools',
'platforms', 'languages-and-frameworks']
options = ['adopt', 'trial', 'assess', 'hold']
choice = None
while choice is None:
quadrant = random.choice(quadrants)
option = random.choice(options)
try:
choice = random.choice(skill_items[quadrant][option])
except:
print('hello i am a mediocre dev')
return choice
def get_phrase(skill_items):
return 'I think that ' + random_skill(skill_items) + ' are ' + random_adjective() + ' and ' + random_skill(skill_items) + ' should strive to be more ' + random_adjective() + ' because'
DeepAI TextGenerator
We will be using a free text generator available at https://deepai.org/machine-learning-model/text-generator.
You can test it out right in the website, however we're going to use the API key. Head over to your profile once you create an account at https://deepai.org/dashboard/profile and generate an API key. The key itself should look like the following:
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
NOTE: To make life easier for you later, I would recommend storing this API key in AWS Parameter store (as I have done in the linkedin-professional repository). This can be done by running the following command provided you have the AWS CLI setup and authenticated
aws ssm put-parameter --overwrite \
--name devopstar-deep-ai-token \
--type String \
--value $DEEP_AI_API_TOKEN
We can now use the API and API key with the following python code. Have a go yourself and pass it in a phrase!
def generate_jargon(phrase):
return requests.post(
"https://api.deepai.org/api/text-generator",
data={
'text': phrase,
},
headers={'api-key': os.environ['DEEP_AI_TOKEN']}
).json()['output']
print(generate_jargon("I love cats alot because they are really fluffy and cool"))
# > I love cats alot because they are really fluffy and cool but I am always at that moment when I feel like the cats are my love and the cats are really great for me. Just as my love for cats has increased. I read that, you can watch my youtube stream that is where I post this.
# How have your cats come back to you in the past month or so in particular?
# Well, during our first months we had been very happy. It was really cool not knowing that what we were doing was supposed to hurt others. In the middle of all these years that we were living and doing such little things in a way to help people, we really, really fell all over themselves. I really felt that maybe I didn't fully realize all there really was. If someone would do something like that, maybe, I was going to get angry or something, and maybe just get really cold. And then I did a little play that, "Maybe I love cats for that reason!" and my cat came back so proud and happy that he really loves me and so happy to let go and have an open mind about the situation. But there was a part of me that really felt that it did not want to hurt us when the day we met, that didn't want her to hurt us, and so if I really think about it and take care of myself (and hopefully I do that all around the house), what kind of things do I want this cat to do
Tech Jargon with DeepAI Text Generator Summary
Congratulations! We've setup some simple code that scrapes ThoughtWorks technology radar the passes that information into the free text generator API at https://deepai.org/machine-learning-model/text-generator
In the next section we will learn how to use the LinkedIn API and setup a process to authenticate and post as our self programmatically.
If you found this particular section interesting or have used it to generate something cool, please reach out to me on twitter and show me @nathangloverAUS.