Background & Overview

Fig. 1 An existing system of distributed robots

Distributed and swarm robotics are rather new fields of artificial intelligence where the tasks normally given to a single robotic device are distributed among a group of robots. This cluster is often able to achieve better overall performance for a large scale task, and creates a form of redundancy for error-tolerance. Typically these clusters do not possess much autonomous intelligence, but rely on simple algorithms to complete their application-specific task. By combining intelligent hardware with these simple robotic devices, capabilities to perform more complex tasks may be provided.

A well-known existing system called Scout, from the University of Minnesota, solves several problems involving locomotion (wheels or jumping), sensor integration (camera, accelometer interface), and includes a simple hardware-communication method. Research on neural-integration in a system of robots, however, has been quite limited.

Artificial Neural Networks (ANNs) emulate the human brain’s ability to learn and recognize patterns. All neural nets consist of neurons as the basic processing nodes, and synapses which store a weight value to connect together the layers of neurons.

Learning can be divided into two categories: supervised and unsupervised. Supervised learning involves use of a feedback-based algorithm such as Backpropagation, where error responsibility is calculated at each node. Unsupervised learning, often using the Hebbian algorithm, finds correlations between input data and divides it into related sets ⁸.

Fig. 2 Simple 3-layer neural network

Fig. 3 SIFT generation of a keypoint descriptor

The SIFT (Scale-Invariant Feature Transformation) ^{15 7} algorithm is used to identify feature points in the video data. Differences between neighboring scales and levels after Gaussian filtering are used to determine whether each point has the potential to be a "feature point". When these points have been identified, a descriptor is generated for each point, invariant to scaling and small 3-dimensional rotations. Object recognition may take place from a SIFT-processed image by finding common feature points between the current image and stored objects in a database.

Triangulation can be used after each time-step to determine the approximate location of each feature point, assuming the distance moved is known. Navigation takes place based on these positions, using the mapping algorithm.

FPGAs (Field Programmable Gate Arrays) are digital integrated circuits which allow logic circuits to be programmed with the VHDL language. A compact method for pulsed neural networks to be implemented in FPGAs has been developed in this project to be used for intelligent distributed robotics.

The use of FPGAs for real-time image processing has been proposed in my previous projects. They are ideal for the implementation of the SIFT image processing algorithm for fast object recognition.

A Xilinx Spartan 3 demo board is currently being used in this project for the FPGA testing.

Fig. 4 FPGA board used for development
[image self created]

An SMIA cell phone camera ²⁴ has been chosen for a real-time high-resolution input for each physical robot. The SMIA standard provides serialization of the video data, which is transmitted at a high speed. Custom circuit boards for camera interface are created. Due to unavailability of sockets and pad spacing too close to solder (0.1mm), conductive tape is used to attach the cameras.

Images are transmitted serially to the FPGA, where a custom VHDL program reads the data and stores it in a RAM module. It is then processed on-chip using the SIFT algorithm to identify feature points.

Fig. 5 1MP SMIA cell phone camera

Fig. 6 Circuit board design for SMIA interface (left), manufactured board (center), camera attached (right)
[images self created]

Fig. 7 System Overview
[image self created]

System overview, showing:

• Camera Interface
• Bayer Format → RGB
• SIFT Transform
• Object Recognition
• Mapping
• 3D Positional Input
• Neural Network Processing
• Communications Algorithm
• Motor Output

To create a prototype hardware-based neural network, a new small-scale stochastic-bitstream neural net design was created. Global signals ¹¹ are used for the sigmoid (tanh) and sigmoid derivative (sech²) functions, as well as a pseudo-random binary-weighted bitstream which is multiplexed by the stored synapse values. Arithmetic operations can then take place with simple gate circuits (i.e. XOR will multiply two bitstreams).

Neurons are implemented using pulse-counters to convert a pulsed input to a binary word. A global sigmoid function is compared to this value to generate a pulsed output. Synapse propagation takes place using an XOR gate for signed pulse multiplication with a stochastic pulsed weight. Synapse outputs are combined together using a dendrite tree: a shift register where inputs are sequentially multiplexed. Multiple dendrite trees may be connected together to expand the number of inputs a neuron can receive.

Fig. 8 Complete Neuron Design
[image self created]

Fig. 9 Complete Synapse Design
[image self created]

Fig. 10 Complete Dendrite Design
[image self created]

A common radio-link is used for communication between robots. In the prototype robot, a Bluetooth module has been used instead to allow simple communication with a computer.

Closed-loop feedback is used in the algorithm to ensure that the neural-generated command is understood correctly. The command may be to provide negative or positive feedback to the other robot, or to share map or object data. Additional commands corresponding to neural defined functions may develop over time.

Fig. 11 Communications Module
[image self created]

“Mirror neurons,” which share the same synapse values as the main neurons, are used to compare the actual result of what is sent to what is expected. No learning takes place in the mirror network, but it rather provides feedback to the main neural network. The loop between the “mirror” and main neural networks forms a recurrent network which can help correct problems with sent data and ensure a standard form of communication is created throughout the system.

Assuming the current position of the robot is known (based on a GPS, accelerometer, and/or known landmarks), identified feature points can be added to the local map as vector lines extending from the camera. Existing vectors can also be confirmed as a 3-dimensional location by triangulation.

A system state is defined when map data is needed. At this point the communication algorithm requests local map data from neighboring robots. This request is repeated to other robots in the system until the map data is acquired or found to not exist.

Fig. 12 Mapping Diagram (showing Feature Vectors, Feature Points, and Obstacles)
[image self created]

To test and demonstrate the performance of a simple distributed robotics system, a C++ two-dimensional environment simulation has been created. The objective of this simulation was to require a neural-based system to learn:

1. To decide on a target object, where target is defined as an object the robot will use to complete an action
2. To navigate to the chosen object and back to the goal when the object has been picked up

As this is a distributed robotics system, at least 2 robots must be used to demonstrate how they can work together to complete the goal. Specifically, one robot is trained from an "expert system" (ideal algorithm) which knows the correct target object that should be used, and the other robot learns from that robot (group-feedback).

The environment consists of balls in the center of the screen and a goal to the left side of the screen behind a line.

Fig. 13 Environment Setup of the Demonstration Program
[image self created]

Balls are automatically picked up upon being touched by the robot, if it is not holding a ball. Robots are aware, through a value, whether or not they are holding a ball. If they are holding a ball, normal physics rules apply upon touching another ball or robot (the second ball is not picked up). Each ball must be dropped off at the left side of the line, where it will disappear.

The demonstration, while relatively simple, adequately demonstrates how robots can work together to complete a goal. It has direct implications to military objectives, where robots must learn to navigate to a specific location when they are needed for an important task, and mining, where robots would have to collect a material and take it to another location.

Minefield clearing is a perfect analogy to this demonstration. As a first step, the mine must be located, either visually or through metal detection. In the same way, the robot locates a ball and chooses one as a target. The robot then must travel to the mine, and either detonate it or move it to another location where it can be detonated.

From this demonstration, it will be determined if this new system based on group-feedback from other robots in the colony is capable of achieving a level of performance comparable to an ideal algorithm. Such an algorithm may not be available to a group of robots in many applications, so group feedback would be an ideal replacement.