Mobile Camera Server

Table of content

Overview
System requirements
Study case
Start MCS
MC Protocol
MC SDK
Matlab toolkit
Licensing
Contact information

System Requirements.

MCS was written in C++ using VC.Net. The program is mainly tested on Windows XP/2000, although it should work with other version of Windows as well.

The application requires Intel OpenCV library and mfc71.dll and msvcr71.dll. All of these files are included in the binary release zip file.

Start MCS

To run MCS together with RCC, do the following steps:

Close ER1 Robot Control Center (RCC) or any applications using the camera.
Start the server.
Select the video hub on the list box then click on 'Acquire' button to select assign video source to it.
In the pop-up dialog, enter the camera ID or (AVI) filename.
Select another video hub and repeat to add more video source.
You can also add/change/release video sources anytime later, even when server is running.
Enter the port number in the box in the low-left corner, the default port is 2810.
Click Start to start server.
The dialog will display the server status with all reachable IPs for the server.
Run ER1 RCC, if need.

Known bugs: the program may work undesirable when the computer switch to standby mode. Most of the time, you can fix problem simply by restart the server (click on 'Stop' and then 'Start' again).

This chapter is for developers who want to write their own client software to work with MCS or want to rebuild/improve the MCS. General users may want to skip to chapter 6 to learn to use our SDK.
This protocol was designed to run efficient on a reliable network. It has very limited mechanism to detect and correct communication data corruption. However, for our research purpose, this does not create any serious problem.

The MC server/client work on request-response basic. All messages, except the image package, are simply a 4-byte opcode; this can be considered as 32bit unsigned interger, sending in 'native' mode (no swapping). The 'image package' contain the opcode MCOP_IMAGE, followed by image header and data.

List of opcodes:

Server side:

MCOP_ACCEPT    0x00080000
MCOP_REJECT    0x00100000
MCOP_SHUTDOWN  0x00800000
MCOP_PING      0x00200000

MCOP_IMAGE     0x00020000
MCOP_NOIMAGE   0x00040000 | SRC_status

which SRC_status are:
SRC_NOT_READY  0x0001
SRC_ERROR      0x0002

Client side:

MCOP_QUERY      0x00010000 | the video source index
MCOP_DISCONNECT 0x00400000
MCOP_PING      0x00200000

Details:

MCOP_ACCEPT: send by server to a new client to accept the connection.
MCOP_REJECT: send by server to a new client to reject the connection. The reason is the server is already occupied.
MCOP_SHUTDOWN: send by server when the server is properly shutdown. The signal helps the client to change is (connection) status.
MCOP_PING: send by server to detect broken connection. On received this package, the client need to send back the PING signal.
MCOP_NOIMAGE: send by server after receiving MCOP_QUERY. The package indicates the query cannot be sastified, either because the video source is not ready (SRC_NOT_READY) or the source corrupted (SRC_ERROR). If the client receives this package, check the video hub on the server.
MCOP_IMAGE: send by server after receiving MCOP_QUERY. This is the only package that followed by data part. The complete package format is as below:
```
< MCOP_IMAGE> < width> < height> <  depth> < nchannel> < origin> < datasize> <           data> 
< 4 bytes  > < 4bytes> < 4bytes> < 4bytes> <   4bytes> < 4bytes> <   4bytes> < datasize bytes>
```
- width: width of the image
- heigh: height of the image
- depth: depth of each pixel (eg 8)
- nchannel: number of color channels (eg 3 for RGB color)
- origin: 1 if the image is flipe upsidedown, 0 otherwise.
- datasize: the number of byte of the image data
- data: image data. Note that the image data is stored in OpenCV format, which align each row to 4-bytes alignment.
MCOP_QUERY: send by client to query an image. The opcode is OR with the video hub number of specify the video source in server. For example, sending 0x00010001 will request an image from hub 1.
MCOP_DISCONNECT : send by client to signal disconnect. This helps server releases resources and becomes openned for new connection.
MCOP_PING : send by client in response for the server PING signal.

Mobile camera SDK

Users can also reuse the 'comm' library to build new application quickly. This library provides a communciation interface via socket for both MCC and RCC, makes it convinient to both control the robot and access the camera remotely.

List of exported classes:

CerClient: a client class to connect to the ER1 robot control centre (RCC). Supported methods:
- Connect - connect to the RCC; returns true if connection is created successfully.
```
bool Connect(const CString& address = "127.0.0.1", int port =9000);
bool Connect(BYTE b0, BYTE b1, BYTE b2, BYTE b3, int port =9000);
```
- Disconnect - disconnect.
```
void Disconnect();
```
- GetLassErrorMsg - to retrieve the last error message. Only one error message is stored at a time and will be overwrite by the next command.
```
						  
CString GetLastErrorMsg();
```
- GetLastCommandResponse - to retrieve the feed back of the last command. Only one message is store at a time and will be overwrite by the next command.
```
CString GetLastCommandResponse();
```
- IsConnected - check if the client is connected to the RCC.
```
BOOL IsConnected();
```
- SetConsole - set the output stream to write the log. Developer can use one of the supported console below or create new one themselves by subclass them. If this is not set, the std output stream will be used. (
```
					 
void SetConsole(CConsole* pConsole);
```
- SendCommand - send a string command to the RCC (provided the connection is established). Returns CMD_SUCCESS (0) if the command is carried out successfully; otherwise, it returns non-zero value. For the each type of the error, refer to CMD_STATUS enum. The last error string can be retrieve using GetLassErrorMsg();
```
int SendCommand(CString cmd)
```
- WaitFor - wait for an action is complete. The client will keep sending 'events xxx' until it received 'xxx done' or 'Error ...', and returns true or false, respectively. The best use of this function is placing it after every issue of 'move' or 'play' command (including Move or Play bellow). Call this more than one may cause a deadlock. In a new version, we may try to embed this call in SendCommand functions.
```
BOOL WaitFor(const CString& action)

Example:
	SendCommand("move 12 d");	//rotate 12 degree
	if (WaitFor("move"))
		printf("Command success");
	else
		printf("Command failed. %", GetLassErrorMsg());
```
- Play - short cut for SendCommand("play ..."). At the moment, ER1 provides 2 different play mode: play phrase (text to speak) or play file (play a sound file). We can use Play to issue either of these two mode. For example:
```
Play(PLAY_PHRASE, "Hello world");
Play(PLAY_FILE,"file.mp3");
```
  Return status as in SendCommand.
- Move - short cut for SendCommand("move... "). There are also a bunch of move varieties:
```
int Move(float distance, CHAR units);  //move a distance unit for the current position

//rotate until it see the object identifie by objectName. 
//This command using the built in recognition software for ER1.
int MoveRotateTowardObject(const CString &objectName);	

int MoveRotateTowardColor(int R, int G, int B); //rotate until it see this color

//Move toward the object. 
int MoveDriveTowardObject(const CString &objectName, float stopDistance = 2,  CHAR units = 'f');	

//Move toward the color.
int MoveDriveTowardColor(int R, int G, int B, int percentage = 10);	
```
  For detail of these functions, refer to the ER1 API. Function returns same as in SendCommand.
- Position - Get the current position of the robot. Function returns same as SendCommand
```
int Position(float &x, float &y, float& angle);
```
- Stop - stop the current action. The method simply send "stop" command to the robot. Function returns same as SendCommand
```
int Stop();
```
- Clear - clear the events buffer. The method simply send "clear" command to the robot. Function returns same as SendCommand
```
int Clear();
```
- Events - Get the status of the current action. The method simply send "events" command to the robot. Function returns same as SendCommand
```
int Events(CString &info);
```
CmcClient - client to connect to the MCServer
- Connect and disconnect:
```
bool Connect(const CString& strAddress, int port);
bool Disconnect();
```
- IsConnected: check the status of the connection. Return true if connection is active.
```
bool IsConnected(){ return m_pClientSocket != NULL;};
```
- QueryImage: get an image from server. Returns true if an image is served. The new image will be stored in pImg structure. srcIndex specifies which camera the client want to access (MCS supports 4 cameras, index from 0 to 3).
```
bool QueryImage(IplImage** pImg, UINT srcIndex = 0);
```
- Set console: set the output stream for log message. A default is std output stream
```
void SetConsole(CConsole *pConsole);
```
CmcServer - server core class to build MCS. Below is highlight of few important methods. Also refer to CmcFrameGrabber
- GetImage - get an image from the specified connected camera. Typically, this function is used to get a new image in response for a client request (see also SendImage), but some time it is helpful to 'peek' the camera resrouces. Function return the image storage pointer (if success) or NULL otherwise
```
IplImage* GetImage(UINT srcIndex = 0);
```
- SendImage - get an image from the camera (GetImage) and send to the client when request.
```
virtual void SendImage(UINT devIndex = 0);
```
CmcFrameGrabber - class to access (USB) camera or AVI files in a uniform way. Use together with CmcServer to build MCS (the MCS may contain one or more instance of this class to handle each of the camera device)
- Attach - attach the grabber to the device (camera or file). Return true if success.
```
//Attach the video stream to camera
bool Attach(int devID, CString name);

//Attach the video stream to AVI file
bool Attach(CString fileName, CString name);
```
- Reattach - try to reattach to the current device. Particular usefull to restart the avi file.
```
bool ReAttach();
```
- Release - release the captured device.
```
void Release();
```
- QueryFrame - return a new image frame from the device. This method simply call the cvQueryFrame with appropriate parameter base on the type of the device (camera or file)
```
IplImage* QueryFrame();
```
CConsole and CConsole family - provide uniform way to output log message to a different type of devices (std stream, MFC CEdit box, CView, etc). To create a new one, simply subclass CConsole and implement protected _Print method.

Matlab ERtoolkit.

This toolkit is a useful tool for researchers. It quitely manages all communications to the camera server and robot control center, provides user a simple and uniform way to access robot and robot camera. The toolkit is available to download from the our project home page . You may also needs to install tcp_udp_ip toolbox ((C) Peter Rydesäter).

Licensing



This software is Copyright © 2004 The Regents of the University of California. All
Rights Reserved.

Permission to use, copy, modify, and distribute this software and its documentation
for educational, research and non-profit purposes, without fee, and without a written
agreement is hereby granted, provided that the above copyright notice, this
paragraph and the following three paragraphs appear in all copies.

Permission to incorporate this software into commercial products may be obtained
by contacting
Technology Transfer Office
9500 Gilman Drive, Mail Code 0910
University of California
La Jolla, CA 92093-0910
(858) 534-5815
invent@ucsd.edu

This software program and documentation are copyrighted by 
The Regents of the University of California. The software 
program and documentation are  supplied "as is", without 
any accompanying services from The Regents. The Regents 
does not warrant that the operation of the program will 
be uninterrupted or error-free. The end-user understands 
that the program was developed for research purposes and is 
advised not to rely exclusively on the program for any reason.

IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO
ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR
CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING
OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION,
EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE. THE UNIVERSITY OF
CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND
THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR
MODIFICATIONS.

Contacts

http://vision.ucsd.edu/mobile_robot/