Rock, Paper, Scissors AI!

Key terms

Tools required:

• Teachable Machine
• P5 Web Editor

Teachable machine

Familiarise yourself with the Teachable machine application. View the supporting videos.

Note: Teachable machine requires an internet connection, Google Chrome on Windows or Macintosh (tablets are not supported) and a webcam.

Image 1: Teachable machine application screenshot

The image above shows the view of a project created in the Teachable machine AI application. On the left, the classes are shown. Once the model is created and trained a preview will be shown.

Privacy and personal information: Discuss the potential misuse of personal images when uploading images of ourselves or friends on websites. Instruct students not to record images of themselves or others via the webcam or uploading images.

The conditions of use for Teachable machine state that images are not stored on external servers if the teachable machine program is closed when completed and the project is not saved. If students close the tab, nothing is saved in their browser or on any servers. View the conditions when saving projects to the cloud on what data is stored and ‘who’ is able to view the model.

P5 web editor

Note: The P5 web editor for this lesson requires an internet connection, browser on Windows or Macintosh and a webcam.

Familiarise yourself with the P5 web editor.

Student accounts will need to be created if importing the Teachable Machine AI model into a JavaScript program. Registration is free.

Note: The P5 web editor was chosen over other similar web editors because it provides a solution to show and use camera input and use the URL from the Teachable machine. A pre-made version is provided to enable students to easily input their model. They can duplicate and remix the code depending on their skill level.

Image 2: P5 web editor screenshot

In this lesson, students will:

1. Describe the process used to train a simple AI using machine learning and describe potential for bias.
2. Create, train and test their machine learning model using an online AI tool.
3. Import their machine learning model into a pre-made JavaScript program and modify the code to engage with the AI.

For more information about learning maps refer to: Mindset, skillset, toolsets

Share the assessment criteria and rubric (in the assessment button) with students.

Show students a real example of an AI that uses computer vision.

Examples include:

Environmental monitoring

• AI + indigenous knowledge in protecting Kakadu National Park

Conservation

• Peace Parks Foundation tackles rhino poaching with Azure Machine Learning and case study

Agriculture

• Orchard Autonomy and Precision Targeting Using RIPPA and VIIPA

Familiarise students with machine learning and the process involved. View this video that concisely defines AI, then defines machine learning through a simple example: What is Machine Learning?

Unplugged activity

Use the context of training an AI to recognise hand gestures for a game of rock, paper, scissors, for students to consider the training data required while also introducing the idea of bias.

Image 3 Rock, paper scissors

The machine learning process

1. Discuss the general process of training an AI for example:
   • labelling and classifying images, (paper, or scissors or rock) Q. What images are required and how many of them in order to achieve a good learning outcome?
   • training the model (the AI uses the data to look for patterns and create fine tunes existing algorithms)
   • testing the model (use new images to test the AI; how well does it predict the correct label and to what level of confidence?)

   Labelling and classifying images

2. Use the worksheet: Labelling and classifying images to consider the range of training data the AI would need to effectively recognise new images of hand gestures.

   Discuss bias

3. Discuss the fact that AI systems are vulnerable to errors introduced during the training process by its human creators. This is referred to as bias. Brainstorm potential areas for bias and how these can be addressed. Examples may include capturing gestures:
   • of only left or only right hand [bias: left or right handed]
   • at the same distance from camera [bias: size]
   • one person’s hand [bias: skin colour and size]
   • same position [bias: positional]
   • only front of the hand [bias: front or back]
   • only one type of background is used [bias: background type/colour]
4. Ask students to explain how they plan to avoid introducing bias in their AI model.

Plugged activity

1. Explain that computers can be programmed to be intelligent, or at least smart.

   Explain that students are going to work in small groups to create, train and test an AI model that recognises hand gestures for a game of Rock, paper, scissors.

   Ask students to apply their learning from the unplugged activity related to labelling and classifying images and avoiding bias.

2. Introduce the tool Teachable machine. Explain that this tool lets you train up an AI application – without having to code – to recognise inputs and match them each to a particular output.
3. Some students may need the process modelled; others may be able to create a model without assistance (for these students go to Learning construction). If required, model how to create, train and test an AI model. Refer to this video explanation or follow these steps:

   

   Image 4: Teachable Machine Application screenshot

   • Edit ‘Class 1’ and label it as Rock. Record images of hand gestures for rockheld in front of the webcam.
   • Edit ‘Class 2’ and label it as Paper. Record images of hand gestures for paper held in front of the webcam. Add a class following the same process to add scissors Class.
   • Once students understand how to create the model, train and preview the model, they can then go to Learning construction to create their own model.

1. Provide students with the opportunity to create their own AI model using Teachable Machine.
2. Students label the classes and record images, providing as many different data types for each class. Consider the background against which the hand gestures are recorded. They can preview and test their model. If required they can make modifications and retrain the model. Once completed they can save their model to the cloud and copy the link so they can import this into another program.
3. Ask students to preview their model and record a screen capture of their trained AI model. As part of the recording ask them to explain how they minimised the chance of bias. Use this as a form of assessment.
4. A completed rock, paper, scissors classifier will look something like this AI model: Rock, paper, scissors. [Requires device with camera enabled].

Options:

You may want to open the project up and provide students with a different context other than the game Rock, paper, scissors. Some students may prefer to apply this programming project to one of the following contexts:

• Create a way to use teachable machine AI model to communicate with a person with a disability.
• Teach a young child to recognise the world around them and read
• Use visual cues such as arrows or hand gestures to control a game for example a maze game such as pac man.

In this part of the lesson students will import their AI model into a JavaScript program using the P5 web editor. Students will need access to a free account to copy the sample code (sketch) provided and duplicate it so they can modify the code in the program.

1. Students will need the URL of the model they created using Teachable Machine. Alternatively students can use this pre-made model

   Note this model is created against a light background; so it has a bias.

2. Provide students access to a computer with a camera enabled and the sample code (sketch)

   Ask students to duplicate the sketch and save it. They can then make changes to the sketch.

3. To get the program working with their own AI model, ask students to insert their own URL copied from their Teachable machine project. Replace the highlighted modelURL as shown.

   

   Image 5: P5 sketch with Teachable machine model URL highlighted

4. Students select ‘run’ to test the program to make sure it is operational and that their AI model is working as expected (correctly recognising hand gestures as Rock, Paper or Scissors). Students can then further develop the program to add a level of game play. The program should output as follows (refer to image).

   

   Image 6: Output: P5 sketch with Teachable machine model

5. The program uses a series of if/else statements to show text onscreen if the label is recognised. Students could firstly modify this section of code.

6. Ask students to use pseudocode to map out changes to the program to incorporate game play. Students code the program and trace errors to fix issues and ensure the program runs as expected. To guide students through decomposing the problem and considering user interface and coding the program refer to the Worksheet planning: AI powered rock, paper, scissors game.

7. For students that need guidance to code the project in JavaScript refer to this online self-paced tutorial: Visual to text coding Lesson 4: Scissors, Paper, Rock

Talk about the bar on the Teachable Machine interface, which displays the AI’s confidence level.

Discuss the scale and percentage. If the bar is fully coloured 100%, the AI is very sure. If the bar is only partly coloured, the AI is not quite sure. Here’s an example from Rock, Paper, Scissors AI model:

Correct: 100% sure	Correct: quite sure	Incorrect
The model is working as expected. The scissors are predicted with 100% confidence.	The model is working as expected. The rock is predicted with 73% confidence.	The model is not working as expected. The model predicts the hand gesture is paper (83%) and less sure it’s a rock (16%). ; sufficient data may not have been used in training.

Discuss:

• When was the AI very sure of its guess?
• When was it not so sure?
• A bias may be experienced if the data used does not contain a diverse range. Examples may include one size, one type of image etc.

Discuss application of these types of technologies for people with a disability.

• How might this application help people communicate who may experience difficulties.
• What images could be used to assist in communicating day to day conversations.
• How might these technologies be used by a person who is vision impared?

AI is the ability of machines to mimic human capabilities in a way that we would consider 'smart'.

Machine learning is an application of AI. With machine learning, we give the machine lots of examples of data, demonstrating what we would like it to do so that it can figure out how to achieve a goal on its own. The machine learns and adapts its strategy to achieve the goal.

In our example, we are feeding the machine images via the inbuilt camera. The more varied the data we provide, the more likely the AI will correctly classify the input as the appropriate classification. In machine learning, the system will give a confidence value; in this case, a percentage and the bar filled or partially filled, represented by colour. The confidence value provides us with an indication of how sure the AI is of its classification.

This lesson focuses on the concept of classification. Classification is a learning technique used to group data based on attributes or features.

Bias can be introduced into the training of an AI. It may be through using a small sample size of data for example one type of image for each classification or the way the data is labelled due to human error or human short comings such training on limited hand types, positions and left or right handedness.

Videos

Teachable Machine 1: Image Classification

In this video, students are shown how to train an image classifier and import the machine learning model into a p5.js sketch with the ml5.js library. The sketch provided uses this sketch as a basis. Some students may be interested in how to code the program from the start.

Worksheets, PDFs

• Worksheet: Labelling and classifying images
• Worksheet: AI powered rock, paper, scissors game