"(Text / Lukas Biewald) in the" made "after the robot can carry out object recognition, the next step is very clear, I want to" make things "a can fly! I decided to do a drone capable of autonomous flight, and as it can recognize faces and respond to voice commands.
Selecting a finished UAV
How is the most difficult part of "Black into" a drone is how to start. My initial choice is to assemble a drone with parts. However, like most DIY projects I have finished, I usually take too much time from zero, it is better to buy a finished product directly. Introduction, I have never flying in my own assembled drones. And buying a finished machine is usually affordable and saves money.
Most drone vendors claim to provide an API interface, but there is no clear and best choice for amateur players. Most of the drones that can provide an API interface that looks likely API have more than $ 1,000. This is too expensive for entry-level players.
But after searching, I found the Parrot AR drone version 2.0 (see Figure 1) is a low-end can be "black" excellent choice for the UAV. The new price is $ 200, but many people don't play again after I bought it, so I spend $ 130 on eBay or less to buy a nice second-hand goods.
Figure 1: UAV collection in my garage. The far left is the Parrot AR drone.
On the flight stability, Parrot AR type as well as more expensive Parrot Bebop type 2 (US $ 550). But called node-ar-drone of node.js client library Parrot AR type is brought very easy to use.
Another advantage of the Parrot Ar type is that it is very raffin and falling. When testing the self-directed flight code, my drone frequently hit the wall, furniture, plants, even the guests of our family, and fall. But it still can still fly normally.
Compared to the robot that "creates" can run on the ground, the "creation" drone is the most uncomfortable thing is that its battery is too short. Basically, it is charged for a few hours, flying for more than ten minutes. So I suggest buy more battery, recurring cycle.
To UAV development program
After practice, I found, Java inherently event-driven language characteristic makes it a very good control of the drone. Please believe me, when drone is flying, there is a lot of asynchronous events. I have written Node.js for a long time, but in the process of this project, I am very impressed.
The last time I was written for the robot to write the program for the robot. Treating C language threads and exceptions is a very distressed thing, so I try to avoid using it again. I hope someone can develop a platform for other robots Java interface, because it makes the robot development process (completely unable to predict what will happen) easier and more fun.
Architecture design
I decided to run control logic on my notebook and run the machine to run the machine in the cloud. This architecture design is lower than that of directly running the neural network directly on the Raspberry Panel. I think this architecture is suitable for amateur projection.
Microsoft, Google, IBM and Amazon offer fast and cheap machine learning APIs. Eventually I chose Microsoft's Cognitive Service API to complete this project because it is the only API that provides customized face recognition. Figure 2 shows the system architecture of the entire drone project.
Figure 2: System architecture of smart drones.
Started working
By default, Parrot AR drone version 2.0 they can provide a wireless network for client access, but this functionality is extremely annoying. Every time you have to experiment, you need to disconnect from the local network, and then go to the wireless network of the drone. The good news is that there is a project called Ardrone-WPA2, very useful, use it you can "black" into the drone, and let the drone join your own wireless network.
Telnet is very interesting on the drone operating system and wandering. Parrot drones use a simplified version of the Linux operating system. The last time you Telnet to a system which is what time? The following command examples to demonstrate on how to open a terminal and log in directly to the UAV operating system.
% / Connect "The Optics Lab" -p "particleorwave" -a 192.168.0.1 -d 192.168.7.43
% telnet 192.168.7.43
To control the flight from the command line
After installing the Node library, the next step is to generate a node.js command line running environment, then you can start to control your drone:
var arDrone = require ( 'ar-drone');
Var client = ardrone.createclient ({ip: '192.168.7.43'});
Client.createREPL ();
Drone> Takeoff ()
True
drone> client.animate ( 'yawDance, 1.0)
If you follow the step by step, you will be here, your drone must have fallen - at least a few times. I haven't given the protective shell of my drone to the machine without countless time until it completely disinsoms, and then I have to buy a new one. I have to say, in fact, Parrot AR type flying in the absence of better protective housing. But this way will make drones very dangerous. Because there is no protection of the case, once no one is hitting, its propeller will hit the object directly and leave a scratch.
UAV flight control from a web page
Develop a web-based control page for drones is quite simple and effective. Use the express.js framework as shown below to build a very small web server.
VAR Express = Require ('Express');
App.get ('/', function (req, res) {
Res.sendfile (path.join (__ DIRNAME + '/DEX.HTML'));
});
App.get ('/ Land', Function (Req, Res) {
CLIENT.LAND ();
});
App.get ('/ takeoff', function (req, res) {
Client.takeoff ();
});
app.listen (3000, function () {
});
I use the following code to send AJAX requests through a button.
function call (name) {
var xhr = new ();
xhr.open ( 'GET', name, true);
xhr.send ();
}
Takeoff
Land
Export video stream from the UAV
I found that the best way to export the video captured on the drone is: establish a continuous connection and send the PNG picture taken by the camera to the web page of my website. By using the AR drone library (see the code below), the web server constantly UAV PNG picture taken by the camera pulled out.
var pngStream = client.getPngStream ();
PNGStream
('Error', Console.log)
('Data', Function) {
Sendpng (pngbuffer);
}
function sendPng (buffer) {
res.write ( '- daboundarynContent-Type: image / pngnContent-length:' + buff
er.length + 'nn');
Res.write (buffer);
});
The image obtained from the UAV for face recognition
Microsoft's Azure Face API system is easy to get started and powerful. You upload your friend's photo to it, this system can identify who they are. It can also guess the age and gender of the characters. I found that the identification accuracy of these two features is amazing high. The entire identification is probably 200 milliseconds. Identify 1000 requests for $ 1.5. For me, this price is quite reasonable for this application. Below is my code, it implements the function of sending pictures to the API to make face recognition.
VAR OXFORD = Require ('Project-Oxford'),
OXC = new oxford.client (client_key);
LoadFaces = function () {
chris_url = "https://media.licdn.com/mpr/mpr/shrinknp_400_400/AAEAAQAAAAAAAALyAAAAJGMyNmIzNWM0LTA5MTYtNDU4Mi05YjExLTgyMzVlMTZjYjEwYw.jpg";
Lukas_url = "https://media.licdn.com/mpr/MPR/SHRINKNP_400_400/P/3/000/058/147/34969d0.jpg";
Oxc.face.facelist.create ('myfaces');
oxc.face.faceList.addFace ( 'myFaces', {url => chris_url, name => 'Chris'});
oxc.face.faceList.addFace ( 'myFaces', {url => lukas_url, name => 'Lukas'});
}
Oxc.face.detect ({
PATH: 'Camera.png',
Analyzesage: True,
Analyzesgender: True
}). THEN (Function (response) {
if (response.length> 0) {
Drawfaces (Response, FileName)
}
});
I used the imagemagick library to make a label for the PNG picture I collected, and the effect is quite good. There is a lot of extensions for this section. For example, use an emotional API to identify the emotions of your face.
Use your voice to control UAVs
The difficulty of conducting voice recognition is not identifying itself, but how to convert the voice stream from the web page running on my local server into Microsoft Speech API can use format. The following code is to implement this function. Once you can save your voice into a mono and sampled in the correct sampling frequency, this speech recognition API can easily identify voice content. This API cost is 1000 requests for $ 4. For amateur applications, it is basically equivalent to it.
RecordRTC is a good library that can be used as a tool for newcomers that are collected with a web page. At the client, I joined the code to save the voice file.
App.Post ('/ Audio', Function (Req, Res) {
Var form = new formidable.incomingform ();
// set allows customers to upload multiple files in a request
Form.Multiples = true;
Form.uploaddir = path.join (__ DIRNAME, '/ UPLOADS');
Form.on ('File', Function (Field, File) {
FileName = "Audio.wav"
fs.rename (file.path, path.join (form.uploaddir, filename);
});
// log errors occurred
Form.on ('Error', Function (ERR) {
console.log ( 'An error has occured: n' + err);
});
// Once all the file upload is complete, send it to the client accordingly
Form.on ('end', function () {
RES.END ('Success');
});
// parsed form data included in the request
Form.Parse (REQ)
Speech.Parsewav ('Uploads / Audio.wav', Function (Text) {
Console.log (Text);
ControlDrone (Text);
});
});
I use FFMPEG tool to reduce the sampling rate of the audio and combine multiple channels into mono to use Microsoft API.
Exports.Parsewav = Function (Wavpath, Callback) {
var cmd = 'ffmpeg -i' + wavPath + '-ar 8000 -ac 1 -y tmp.wav';
Exec (CMD, Function (Error, stdout, stderr) {
console.log (stderr); // command output is in stdout
});
PostTooxford (Callback);
});
Although the function I develop is these, but it can still be extended. For example, use Microsoft's text variable speech API to let drones talk!
Develop independent search path
I used the Ardrone-Autonomy library to develop an autonomous search path for drones. During this process, I have bumped the drone in the living room in numerous times. Finally, my wife is very "polite", I suggest I go to the garage to continue my project, because there is not much thing that can hit it. But the garage is a little small, making the control space.
Figure 3: Try the drone in my "laboratory".
After I can have a bigger experimental space, I will try more intelligent search algorithms. However, now I still will only let the drone to take off and rotate, and search for people and identify that the enemy is still a friend.
Var autonomy = Require ('Ardrone-Autonomy');
Var mission = autonomy.createmission ({ip: '10.0.1.3 ', framerate: 1, imageSize:' 640: 320 '};
Console.log ("Here We GO!")
mission.takeoff ()
.zero () // put the current state as a reference reference
.altitude (1)
. Tasksync (Console.log ("Checkpoint 1"))
.go ({x: 0, Y: 0, z: 1, yaw: 90})
.tasksync (Console.log ("Checkpoint 2"))
.hover (1000)
.go ({x: 0, Y: 0, z: 1, yaw: 180})
. Tasksync (Console.log ("Checkpoint 3"))
.hover (1000)
.go ({x: 0, Y: 0, z: 1, yaw: 270})
. Tasksync (Console.log ("CheckPoint 4");
.hover (1000)
.go ({x: 0, Y: 0, z: 1, yaw: 0
.land ()
final effect
Look at the video below. I let the drone take off and go to my friend chris:
in conclusion
After everything is configured, you can control the drone through the API, get the video picture captured, all this is cool! As new image identification techniques are available, there may be more and more applications. For example, let the drone brush the wall according to the floor plan. Although the Parrot drone is not designed to be safely flying in a narrow space (such as my house), as the drone is getting more and more skin, the price becomes lower, I believe that the real useful application will enter Duration.
Microsoft's cognitive service cloud API is quite easy to use and cheap. Initially I am more worried that the picture taken by the wide-angle camera used by the drone will affect the accuracy of the face recognition, and the noise of the propeller may cause interference to speech recognition. But overall, these two APIs behave far more than my expectation. At the same time, the delay is also lower than my expectation. From the perspective of architecture design, running machine learning in the cloud seems to be a strange choice, but it may become the architecture selection of many applications in the future.
Search "" Love Board "", pay attention, daily update development board, intelligent hardware, open source hardware, activity and other information can make you fully master. Recommended attention!
[WeChat scanning picture can be directly paid]
Related Reading:
AMD ZEN will be able to fight 8 core I7 intel is not panic? "
Our other product:
