July 25, 2021     2min read

LinkedIn Professional - Generating Tech Jargon with DeepAI Text Generator

LinkedIn Professional - Generating Tech Jargon with DeepAI Text Generator

Get the code for this post!

t04glovern/linkedin-professional

Table of Contents


LinkedIn Professional - Generating Tech Jargon with DeepAI Text Generator

In this post we're going to start looking into how we would be able to create AI generated text sentences using some simple and free online services. This includes the code required to pull down random skills from the ThoughtWorks technology radar website as well.

I've updated the architecture diagram below to include the components that we will be touching for this portion of the guide.

LinkedIn Professional Architecture - Part 1
LinkedIn Professional Architecture - Part 1

Prerequisites

To deploy this project yourself you will require a couple things to be pre-setup:

ThoughtWorks technology radar skill scraping

Let's get started writing some basic python code for scraping the skills from the ThoughtWorks technology radar website. To start with let's take a look at the website and work out what we need to do to extract the data.

ThoughtWorks technology radar base website
ThoughtWorks technology radar base website

We can see that there are 4 pages for each of the sections and each are available at a different subpage of the website.

ThoughtWorks technology radar quadrant source
ThoughtWorks technology radar quadrant source

I wrote the following code immediately that we will be able to use shortly.

radar_base_url = 'https://www.thoughtworks.com/radar/'
quadrants = ['techniques', 'tools', 'platforms', 'languages-and-frameworks']

Each of the subpages have 4 sections; adopt, trial, assess and hold. So I we're going to install a library called Beautiful Soup that can handle website parsing programmatically. This library can be installed with the following

pip install beautifulsoup4==4.9.3 requests==2.25.1

Then I wrote the following code to parse out the skills from HTML, and then a function that leverages that other function to select a random skill

from bs4 import BeautifulSoup
import os

radar_base_url = 'https://www.thoughtworks.com/radar/'
quadrants = ['techniques', 'tools', 'platforms', 'languages-and-frameworks']
skill_items = {}

def skills_from_html(html_target_radar):
    skill_group = {}
    options = ['adopt', 'trial', 'assess', 'hold']
    for option in options:
        radar_option = html_target_radar.find(id=option)
        items = radar_option.find_all('li')
        skill_group[option] = list(map(lambda x: x.find('a').text, items))
    return skill_group

def get_skill_items():
    # Only update the list if empty
    if not skill_items:
        for quadrant in quadrants:
            target_radar_url = radar_base_url + quadrant
            target_radar_page = requests.get(target_radar_url)
            target_radar_soup = BeautifulSoup(
                target_radar_page.content, 'html.parser')
            target_radar = target_radar_soup.find(id='responsive-tech-radar')
            skill_items[quadrant] = skills_from_html(target_radar)
    return skill_items

Awesome! we have a way to pull out skill lists now. I've also included a nice little way to generate a sentence in a structure that we can submit to a sentence generating.

def random_skill(skill_items):
    quadrants = ['techniques', 'tools',
                 'platforms', 'languages-and-frameworks']
    options = ['adopt', 'trial', 'assess', 'hold']
    choice = None
    while choice is None:
        quadrant = random.choice(quadrants)
        option = random.choice(options)
        try:
            choice = random.choice(skill_items[quadrant][option])
        except:
            print('hello i am a mediocre dev')
    return choice

def get_phrase(skill_items):
    return 'I think that ' + random_skill(skill_items) + ' are ' + random_adjective() + ' and ' + random_skill(skill_items) + ' should strive to be more ' + random_adjective() + ' because'

DeepAI TextGenerator

We will be using a free text generator available at https://deepai.org/machine-learning-model/text-generator.

You can test it out right in the website, however we're going to use the API key. Head over to your profile once you create an account at https://deepai.org/dashboard/profile and generate an API key. The key itself should look like the following:

xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

NOTE: To make life easier for you later, I would recommend storing this API key in AWS Parameter store (as I have done in the linkedin-professional repository). This can be done by running the following command provided you have the AWS CLI setup and authenticated

aws ssm put-parameter --overwrite \
    --name devopstar-deep-ai-token \
    --type String \
    --value $DEEP_AI_API_TOKEN

We can now use the API and API key with the following python code. Have a go yourself and pass it in a phrase!

def generate_jargon(phrase):
    return requests.post(
        "https://api.deepai.org/api/text-generator",
        data={
            'text': phrase,
        },
        headers={'api-key': os.environ['DEEP_AI_TOKEN']}
    ).json()['output']

print(generate_jargon("I love cats alot because they are really fluffy and cool"))
# > I love cats alot because they are really fluffy and cool but I am always at that moment when I feel like the cats are my love and the cats are really great for me. Just as my love for cats has increased. I read that, you can watch my youtube stream that is where I post this.

# How have your cats come back to you in the past month or so in particular?

# Well, during our first months we had been very happy. It was really cool not knowing that what we were doing was supposed to hurt others. In the middle of all these years that we were living and doing such little things in a way to help people, we really, really fell all over themselves. I really felt that maybe I didn't fully realize all there really was. If someone would do something like that, maybe, I was going to get angry or something, and maybe just get really cold. And then I did a little play that, "Maybe I love cats for that reason!" and my cat came back so proud and happy that he really loves me and so happy to let go and have an open mind about the situation. But there was a part of me that really felt that it did not want to hurt us when the day we met, that didn't want her to hurt us, and so if I really think about it and take care of myself (and hopefully I do that all around the house), what kind of things do I want this cat to do

Tech Jargon with DeepAI Text Generator Summary

Congratulations! We've setup some simple code that scrapes ThoughtWorks technology radar the passes that information into the free text generator API at https://deepai.org/machine-learning-model/text-generator

In the next section we will learn how to use the LinkedIn API and setup a process to authenticate and post as our self programmatically.

If you found this particular section interesting or have used it to generate something cool, please reach out to me on twitter and show me @nathangloverAUS.

devopstar

DevOpStar by Nathan Glover | 2021