Tag: Driverless cars

Troglodyte Driverless vehicles 5

 

SYSTEM AND METHOD FOR PREDICTING BEHAVIORS OF DETECTED OBJECTS

“Majority of the description text could be condensed to: autonomous vehicles should mimic the behavior of human drivers.”

The purpose of Project Troglodyte is to hunt for bad patents and to show what went wrong. For more information, please see the web page.

This patent is the fifth in a series of Google autonomous vehicle patents/applications analysed to get an understanding of the level of their inventions and the state of the autonomous car project.

 

Figure 1.

 

TIER 1: SUMMARY

It appears that the main purpose of the application is to expose a lot of prior art in one document, to make sure that it is easily found and public. This conclusion is made as there are about 12 000 words in the description but the claims only touch a very small part of it and much of the description text is obviously obvious to anyone skilled in the art, or misquoting from the application: “…understood by those of ordinary skill…“.

The actual idea that protection is sought for is changing how the vehicle is controlled based on detecting an object, classifying the object and based on the classification predicting the behavior of the object. And as Google is involved, creating a massive cloud based database of said behavioral data and sharing it around.

Majority of the description text could be condensed to: autonomous vehicles should mimic the behavior of human drivers. The description explains that processing of the object related information can be done at a location external to the car, this is also mentioned to be possible for the processing related to vehicle control decisions. This might open an interpretation that any controlling of traffic based on information originating in behavior prediction of single vehicles would fall under the protection of this patent. It would mean that any system arbitrating route decisions between vehicles to lessen traffic jams might need to license this.

Being able to predict behavior of nearby objects based on common experience is a valuable feature and will make traffic flow faster and safer. It isn’t mandatory for every autonomous vehicle though and thus wouldn’t likely block competitors from entering the field.

 

TIER 2: AVOIDING LICENSING

It seems that the possibility of using predictions of object behavior of nearby objects observed by other vehicles (or systems) is not mentioned. This would be useful in case large objects create shadows preventing direct observation. Using direct or network based vehicle to vehicle communication might be bandwidth limited in transferring the whole awareness of another vehicle. It would also be wasteful in use of processor resources as the same data would have to be analysed several times, so it would be prudent to  transfer only information deemed important for other vehicles.

If the classification scheme is left out it makes it possible to implement simpler threat prediction based on observed speed and direction. It would still be possible to use context dependent database to predict that for example vehicles in the left lane are more likely to transfer to the right lane during a certain time window at a certain time. This would likely be good enough for autonomous vehicles, but it would be less optimal as the classification scheme will lower the number of times the vehicle needs to alter it course to accommodate other vehicles. Vehicle without the classification ability would likely appear more selfish but if all vehicles are eventually  automated this would have less of an impact as it would now when all the drivers are humans.

 

TIER 3: TECHNICAL ANALYSIS

As stated above, major part of the description just portrays how humans approach driving. Context sensitive behavior prediction of classified objects is what humans are good at. But sharing the accumulated experience between humans is cumbersome. With this invention autonomous vehicles could share automatically on a massive scale. The invention here is not mind boggling, but they usually aren’t. I didn’t do a proper prior art search so it could already be out there, but generally this type of thing (essentially an optimization of a more general approach) is less likely to pop up in science fiction than most of the other stuff in the description.

The description is mostly useless. If the patent system worked, most of the stuff would have to be cut. If there is need to create prior art to stop trolls, write a white paper and publish it somewhere. For the price of a patent attorney it is probably possible to buy enough space in some regional newspaper to show the whole 12 kwords. On the other hand the description of the invention itself is very shallow in detail. Much more should have been given regarding possible ways to implement it, how to handle false identifications, how to handle different sensing abilities, who is responsible if bad data leads to accidents etc. Of course if the patent office doesn’t require this then it would be foolish for anyone to give it. Writing it down might have given a good patent engineer the chance to claim more and could have made this patent more valuable.

The claims only use a small portion of the text but cover that part fairly well. They are almost understandable, although the last one is complex enough that reading it requires more uninterrupted concentration than is usually available when the kids are around.

Troglodyte: Driverless vehicles 1

“This is solid engineering but I didn’t get the “hey this is clever” reaction which is a sort of indicator for inventiveness.”

The purpose of Project Troglodyte is to hunt for bad patents and to show what went wrong. For more information, see the  web page.

 

TRAFFIC SIGNAL MAPPING AND DETECTION

I have been interested in driverless vehicles for years and I like to read patents, so why not combine the two and share my thoughts. This is a short analysis of a Google US patent application, you can find the original here.



Figure 1.

 

TIER 1: SUMMARY

Contents in one sentence: mapping of traffic lights to enable real time status detection of those lights by vehicles. The description is not limited to automated vehicles, rather what is described is a general system of collecting location and orientation information of traffic lights and use of the results in the form of a map to enable detection of the state of said lights at a later time. When reading the patent I got the feeling that this is solid engineering but I didn’t get the “hey this is clever” reaction which is a sort of indicator for inventiveness.

The claims are not limited to a large database that all the vehicles would use to get the traffic light location information, it would also cover saving the same information in the cars own systems. I.e. it would not be possible for your car to store information of where the lights are on the routes you often drive, that is unless it had a licence from google to use.

This application seems to be intended to create difficulties for anyone who wishes to create a map of traffic lights for the purpose of guiding automated vehicles. It could give Google an advantage as creating and especially maintaining such a map by other means could pose some difficulties. Secondly there is a possibility that if it is granted in its current form Google might be able to prevent others from using information of traffic light location to help real time detection of the light state. As part of a portfolio of automated driving patents it could have some value, although there are other methods of getting the same information to car systems. The nice thing about the mapping idea is that it requires very little liaison with authorities maintaining the traffic system.

If the world is going to move to fully automated vehicles traffic lights are probably not needed in the sense they are currently used. Thus this patent would only be useful during a transition period, but the transition could easily be longer than the duration of the protection a patent offers

It is a bit scary that the description gives information on known triangulation and image recognition technologies as this might open a lot of trolling opportunities when the claims are widely interpreted.

TIER 2: AVOIDING LICENSING

An alternate to the map described in this document could be to determine when the light changes occur and from several of these time stamps create a state machine with transitions at known times. The timing information could then be distributed from the cloud to vehicles on their way to the same intersection. Knowing which light should be active would make it a bit faster to find the fixtures from larger images that result from not knowing the location of the traffic light. This would not need a map of locations of the traffic lights, it would only be a map of traffic light state machines with much more lax accuracy requirements . Vehicles could also take advantage of this information to optimize speed, thus reducing maximum accelerations and likely speed and therefore also lowering the likelihood of accidents or at least make the results less severe.

In the discussion of background it is mentioned that efforts have been made to develop systems that use radio transmission of traffic light state but the infrastructure investment is seen as an obstacle. Why not use existing radio infra to transmit the information? A scheme where the traffic light state is available on the net and read through a cellular data connection would need less tampering with the infrastructure. This of course only works in places where the traffic lights are centrally controlled.

The obvious bypassing technique is to not have a map of traffic lights and scan for them continually in the same general direction where drivers look for them. The cost of this surely will become lower as more and more computational power becomes available. One way of avoiding the use of a map could be to ask the vehicles in front of you: where did they find the light. Vehicle to vehicle communications is likely going to be ubiquitous before driverless cars, so there is a good chance that at some point only software development is required to implement exchange of the information. Getting the advantage of making the map in good weather and lighting conditions is more difficult to achieve with other methods.

TIER 3: TECHNICAL ANALYSIS

The description offers a fair explanation of the intended system though most of the details must be known technology. Triangulation and related methods are a widely used technology as is identification of objects with roughly known characteristics from images. Saving the location and characteristics of the identified objects in a database that can be called a map is likewise a well known approach for representing data in an accessible format.

At one point there is talk about triangulation and at the same time about using a sphere to determine which labels (i.e. location of traffic light in an image) will be associated with each other. Later it is mentioned that this determination may be part of an analysis of an image sequence, using a template to follow the traffic light in the consecutive images. At some point a first location determination needs to be done to get a center for the uncertainty sphere. A circular logic seems to be in use.

Knowing the location of the traffic lights before detection will likely lower the false interpretation rate especially if the conditions at the time of detection are difficult: for example there is fog, difficult lighting conditions for the cameras or heavy rain. It will also lower the computational intensity and thus lower the cost. This may be a significant advantage.

Several different types of traffic lights are in use around the world but the description is very light on how these could be identified. This is especially a problem in the map creation phase as it is crucial to make correct interpretations. If successful this effort would make the map more valuable as the vehicle would only need to identify the traffic light fixture and could use the map to identify positions of different types of lights (arrows etc.)

Value of the description is lowered by some pretty standard patent speak, for example why draw a flowchart and then say the boxes can be in any order? Is it because the examiners like flow charts and not bullet point lists? In the current form it reduces the value of the flow chart to zero, all of the text it contains is already in the description.

While creating a system that does the mapping is certainly not a trivial undertaking I find it difficult to see what is the inventive step. The mix of patent speak and technical writing in the description could effectively hide it but I would argue that the description is pretty much what most skilled in the art would try after they realise that real time identification takes too much processing power and results in too many errors.

Generally the claims seem to refer to the text in the description part and to the images.

The claims curiously omit other than color based identification of the lights (for example 20), embedded image, shape, frequency etc. could also be used for identification. With LED traffic lights it might even be possible to do a software upgrade to make them blink fast enough to show this in a row read camera sensor. Further, the color identification scheme is lacking in detail. Color is mentioned but not intensity, if the position of each intended signal is known, then it is possible to identify the state from just the relative intensity of the three indicators. Further, depending on the color of the surrounding light being reflected from the traffic light and the color of the light emitted from the traffic light there might not be a large difference in color between the on and off states anyway.

 

Translate »