I. INTRODUCTION
In the last few years, the automation of control system focused on transport facilities has received widespread attention. For implementation, both traffic control system and vehicle control system require the traffic data or road data. The ultimate goal of traffic control system may be considered to give comfortable condition to drivers with no traffic jam nor difficulties in driving. The ultimate goal of vehicle control system is to get information for driving, which is useful to decide how much accelerate, brake or steer.
Since it is difficult to decide "object" in the scene is vehicle or not, the data should be collected manually. Manual collection is some kind of recognizing way, so accuracy is very well. But when person becomes tired the accuracy of data becomes bad.
On the other hand, traditional sensors have been used to collect those data. If the condition is controlled, for example if cars move just under the sensor with assumed direction and assumed speed, loop coil can accurately detect that the car is under the sensor or not. Although machinery works forever, unfortunately sometimes the car through so slowly under the sensor or the car moved not assumed direction. At this case, the result of the sensor is not correct.
When the image sensor is used, many algorithms can be applied to detect the presence of car, to detect the car motion, etc, as same function as loop coil. The advantage of image comes from that image includes information of spread area along the road, called spatial information, and image sequence includes information of the loci of the object, called temporal information. Image processing can achieve various requirements with high accuracy.
Recent progress of hard ware and soft ware may prepare the communication channel between the road and the car. As the result, the transmitter located beside the road can tell to the driver where the car is. And the transmitter located inside the car can tell the identification code to the road.
If various sensors or algorithms are used, the results should be selected or accumulated. For Then system can estimate future traveling from the history of past traveling time from here to there. Past traveling time is measured several ways. One is measuring from the length of traffic jam and average velocity at fixed point of the road. Other is measuring from car identifications at entrance and exit of the road.
Fig.1 image sensor in system
This paper is concentrated to the measurement based on image sensor. In addition to the same function as loop coil, image processing is required to cope with some difficulties on the surface of the road or besides the road, the car density at far from the driver, loci of lane changing car, etc.
The most problem of image processing is the intensity of the picture element not always correspond to car presence, because of ambiguous brightness, shadow existence, different color of cars, and so on. So we have better to consider successive processing for accurate measurement, as shown in Fig.1.
II. REQUIREMENTS
A. Requirement for traffic control
The basic requirement of traffic control, regulation and caution are as follows.
1. Measuring traffic flow velocity and counting cars through the fixed point.
The role of image processing is detecting cars as moving object.
2. Counting cars at assigned place, for example at the parking lot or at the place of parking inhibited.
The role of image processing is detecting cars by pattern matching.
3. Measuring traffic flow in intersection or in merging section.
The role of image processing is detecting loci of cars, after that counting cars and measuring car velocity.
Sensing should be done at many fixed point of the road.
Traffic flow in linear lane, in branch or merge section and in intersection, presence of car stopping, etc are sent to the control center. Usually traffic control, regulation or caution action is done manually by traffic officer at control center with gathering the basic requirement.
Some case of traffic caution is done automatically by system as to show message on presentation board for drivers.
Image processing technology has introduced following requirement
4. Detecting car moving abnormally, caution is presented for detected car.
5. Detecting car meeting accident, caution is presented for following cars.
E. Requirement for vehicle control
B. The basic requirement of vehicle control is as follows.
1. Finding driving lane.
The role of image processing is reconstruction of horizontal and vertical road figure.
2. Measuring distance to following car.
The role of image processing is detecting cars in the following lane.
3. Detecting obstacles in the driving lane.
4. Recognizing view condition such as fog, smoke, rain, etc.jh
5. Recognizing road surface condition such as roughness, iced, etc.
C. Requirement for driving assistance
System for vehicle control can be useful for driving assistance. It can be called auto copilot. The basic requirements are as follows.
1. Observing environment around the car.
2. Observing road surface condition.
3. Observing drivers behavior.
4. Estimating traveling time.
5. Estimating coming parking lot is crowded or not.
6. Navigating to the destination.
III. SENSORS
A. Sensor for traffic and vehicle control
1. Manually counting.
It is ideal but not accurate.
2. Laser radar.
It gives distance to object.
3. Supersonic wave.
It gives distance, but difficult to imaging.
4. Image processing.
It gives information of area, but needs processing.
5. Measuring using communication.
It needs amount of infrastructure.
B. Sensor for spatial purpose
1. Linear traffic flow air pipe, supersonic wave, loop coil, radio beacon, radar, ID plate, ID card.
2. Traffic flow at merging section or intersection ID-plate, ID-card, contact sensor, loop coil.
3. Driver’s behavior (driving way, fatigue, dozing) brain waves. ECG, eyeball motion, face image.
4. Driving condition (speed) speed meter, acceleration meter, Doppler radar.
5. Position of the car GPS, radar, reflector, and beacon.
6. Condition of atmosphere.
Physical sensor (vibration, friction, slippage, water, temperature, bumpy).
C. Video camera
Video camera has possibility to use most of requirement.
IV. IMAGE PROCESSING
The image processing algorithm uses an image sequence to measure the vehicle motion information. The block diagram of image processing is illustrated in Fig.4,
A. Original image
A video camera gives 2 dimensional intensity image f(s, y, t), usually it is called original image. where t means k*?t , and ?t is sampling interval. As progress of hardware and software, range or distance image z ( s , y , t ) become able to be gotten in red time.
B. Pre processed image
The first step of image processing is to find which picture element corresponds to vehicle.
But background image changes, because of time varying environment, so much noise remains on vehicle as well as other part. It has been many studies to get adaptive background image, b(z, y , t ) described after.
b. Binarizing
Binarizing of original image or of differential image gives vehicle region or the boundary of the road.
c. Processing
The second step of image processing is to find characteristic features which depends on required item.
a. An existence of car detecting
Difference and binarizing of original image presents changed pixel. Usually changed pixel corresponds to vehicle.
b. A loci of car detecting
Tracking of changed pixel frame by frame gives loci of the region. Loci of changed pixel correspond to vehicle more reliable than that comes from processing of one frame.
 |
D. Item of feature
a. Basic item
A. basic items are as follows:
1. To detect the velocity of the car
2. To detect presence of the car
3. To find a type of car.
4. Number plate recognition
b. Processed item
Processed item explained before are corresponded to required item. It can be shown after some processing of the results of basic item. For example, traveling time can be calculate from data of loop coil, and also from the results of number plate recognition at entrance and exit for the car identification.
V. EXAMPLES
A. Processing within fixed window
When the camera is located upon the road in a tunnel, system can detect existence of the car or passing through of the car at this region, by processing within fixed window, which corresponds to fixed area on the real road surface.
By detection of shadow lies on road surface just under the vehicle or detection of brightness of the tale lamp, presence of the car detected. In tunnel the light condition is well, so temporal difference works well to find the pixel corresponding to moving car.
b. To find parking violation
By difference between sequential image and background image, system is respected to be able to know parking car beside the traveling lane. At this case, the light condition is not well, so some kind of averaging is required.
B. projection
Projection is useful to reduce some kind of noise. Two examples will be shown.
a. To find accidental stopping
After projection gives projx[y] frame by frame, it can be considered temp spatial map as shown in Figure 13(b).
It is easier to analyze in Figure 12 than to analyze in Figure 14. In Figure 14(a), case of traffic accident is shown.
In Figure 14(b), case of traffic jam is shown.
 |
b. To detect the type of lane mark
Usually, the first step of vehicle control is to extract lane mark from the road image. It is easy to extract from continuous straight edge, but difficult to extract from dotted lane mark or zebra zone, where the end of branching lane or beginning of the merging lane. At this case, it is convenient to detect the type of lane mark before edge extraction. An analyzing of the projection shown in Figure 15 can separate by type of lane mark, without any difficulty.
C. Tracking
Tracking the moving region extracted from temporal difference may give loci of moving object. Short loci can be neglected as noise, and short gap can be connected easily using the motion history. An example is described.
a. To count the car in parking lot
At the parking lot located in resting facilities of the expressway, counting car is required. The method to count cars by tracking the moving objects for the whole area of the outdoor parking lot is proposed [3][4].
It is effective to use time differential images to extract moving objects from stationary objects. However, a moving object can often be taken as many regions in the differential image.
Therefore, the method to match between two differential images and to link moving regions, as shown in Fig.17 (a), allow the system to connect the reliable moving regions. And iterative process makes locus of the moving object.
Then, each locus is discriminated whether it is a car or not. If the locus is considered as a car, counting the number of parking cars in the counting area can be done.
Fig. 17 Tracking the moving regions
This system is implemented by combination of personal computer with a high speed image processor, and accomplished real time processing, 0.2 second per frame, by preprocessing on a high speed image processor.
D. Rule based recognition
When system knows the contents of image and rules of content construction, the knowledge can help the image processing to understand the structural constituents. An example applied to 3D reconstruction of road is described.
a. 3D reconstruction from image seen from vehicle
The highway is constructed under law of road construction. Which shows the lane marker of the road is drawn orthogonal by both edge of road segment S. At this case rules are as follows;
The width of the S is 2W and tangent of the lane maker, at both edge of S, named ", and T, are always orthogonal to S, as shown in Figure 18. The 30 reconstruction of road can be done using the pair of lane markers in a road image, and correspondence between P(X, Y, 2) in 3D coordinate and p ( z , y ) in image coordinate by perspective projection. Fig.18 also shows lane mark extracted by 3D information of road.
When it is assumed that image point pair p 1(x1 ,y1 ) are the both edge on the same road segment S, as shown in Figure 18, eq.(1),(3)and(4) give 30 coordinate pair P1(X1,y1,z1) Pr,(Xr,Yr,Zr,).
If the point p (z, y) does not correspond to pt(xt,yt) then it gives non zero value of S*Vt. Scanning y from the lowest y to the highest y (shown in Fig.19(a)) gives increasing function of S*Ti (shown in Fig.19(b)), and the corresponding point y, can easily be found.
After that, the point p1,(z1, y1) on the left lane mark can be changed from the lowest y to the highest y, every point ( x1,y1) gives a corresponding point pr( xr, yr) on the right lane mark as shown in Fig.20 (a). Amount of corresponding points make 3 0 shape of road shown in Fig.20 (b).
VI. PROBLEMS
A. Dynamic range of video cameraM
Conventional TV camera has dynamic range of 0 to 5*102cd/m2 when outdoor image processing require dynamic range 0 to 104 cd/m2. Using automatic iris process, usually used, gives image as shown in Figure 21.
To evaluate the intensity of temporal average for pixel by pixel gives suitable condition of exposure or shutter speed.
B. View angle of video camera
Usually view angle of video camera is about 30", sometimes it is too narrow. When fish eye camera is used, the view angle is extended to 180" or more, however it serves the non-linear image as well as the low resolution image.
When the fish eye camera is used and vehicle moves straight, image sequence including front view, side view and rear view of vehicle can be gotten by only one fixed camera. And 3D model of the vehicle can be constructed.
C. Estimating background image
Difference of background image is useful to detect vehicles. But background image itself changes slowly or sometimes changes rapidly.
To evaluate the intensity change of each pixel, the state of each pixel should be discriminated correspondence to 1) stationary background, 2) moving object, 3) moving object temporally stopping and 4) stationary background appeared after moving object left. Such kind of decision has to include errors. Then some kind of reliability should be introduced.
D. Binarized
It is well known that low threshold value gives a large amount of patterns but not reliable region, and high threshold value gives few patterns but reliable regions.
To decide the globally optimal threshold value is difficult, but several kind of binarizing strategy may be used suitable processing.
VII. CONCLUSION
Image sensing techniques in vehicle and traffic control are discussed in this paper. First, the basic requirements were defined, and many kinds of sensors for traffic estimation were classified.
Practical applications using image processing were tested with several techniques; 1) To find moving car and 2) to find parking violation based on the fixed window, 3) To find accidental stopping and 4) to detect the type of lane mark by projection, 5) To count cars in parking lot by tracking, 6) To recognize the 3D reconstruction from the vehicle-fixed camera by rule based recognitions. Finally, problems in image processing, such as 1) dynamic range of video camera, 2) view angle of video camera, 3) estimating background image, and 4) binarization, were considered for the applications.
|
