Here is link of FRA research on ATO. Note that a lot of different cameras were tested.
Automated Train Operations (ATO) Safety and Sensor Development (dot.gov)
I didn’t get very far into this article until I was outraged right in the first paragraph where they began to discuss SP (sensory platform).
To mean, I will always equate “SP” with the late, great Southern Pacific.
May she rest in peace! Gone but not forgotten.
Regards,
FMC
Thanks for a more definitive post link. I hope all interested in this topic will read carefully and with an open mind.
The article seems to suggest that several different sensors will need to be employed to cover a range of possible conditions. Layering the images together to provide a composite will require a pretty powerful computer, as well as calibration of the sensors so they see the same thing in the same place.
I would imagine a dedicated pod of sensors would be involved. Front of the loco? On the roof?
That said, there’s no reason such integration can be done.
The challenge will be identifying and programming the actionable objects of interest so appropriate actions can be taken when they are detected.
That would include errant autos and herds of pronghorns.
First thing that came to mind was redundancy in sensors. For example, having two or more separate forward facing cameras, so a bug splat on one camera doesn’t blind the computer. Multiple detection modalities should be mandatory, mm wave radar data is a lot easier to process to get a 3D image of what’s up ahead. Integration with lineside sensors would help, especially in populated areas.
Note that this only barely scratches the surface – it is limited to the raw sensor modalities. The real ‘interest’ in this space is in the discipline we called ‘sensor fusion’ – this has taken a leaf from successful user-interface technology to include a relatively enormous number of parallel devices and sources for the desired redundancy.
I assume that many of the “SPs” being developed for autonomous road vehicles will be quickly costed-down for production and the ‘technology’, if not indeed available as both modules and cores OTS, will then be adaptable to the special conditions necessary for railroad service.
I scanned through the PDF, but did not see much explaining why various sensors are needed to see such a large area. Trains can’t stop for emergencies anyway. Even attempting to stop is too risky for the train. I did notice the point that sensors will help prevent trains from running over herds of animals. I think the railroads will just want to use what Rio Tinto uses to watch the track ahead.
What I would like to see is a full diagram of what will be included in the sensor “platform” and what it will cost to equip one locomotive with it.
Why the Need for Such Elaborate Sensing Ability?
With self-driving trains, there is no need for the train to “see” where it is going. That need is for self-driving road vehicles because they need to see in order to steer. On the contrary, trains do not need the aid of human or artificial intelligence in order to steer because they are self-guiding. Also, unlike road vehicles, trains do not need to see in order to avoid collisions because the collision protection is already provided PTC.
So, why the emphasis on ultra-high ability for automatic trains to sense and assimilate all elements of the unfolding scene ahead when it is not necessary? Where it is necessary is with self-driving road vehicles that are not self-guiding and do have to detect and react to other vehicles. This need by self-driving road vehicles to see and assimilate everything in their view is their biggest technological challenge, and may never be adequately resolved to permit them to be practical. With road vehicles, drivers must be ready to take many different evasive actions to avoid collisions and keep the vehicle on the road.
So why is the FRA (in the link of the OP) advocating the elaborate platform array carrying 8 different modes of detection sensors? Why, for instance, does the automatic train need to know whether it is foggy, snowing, or if animals are on the right of way? It seems like massive overkill to address a problem that does not exist for trains, but does exist for road vehicles.
I believe the explanation for wanting this intense forward vision for drive
I wonder, when trains become driver-less, if railfans will want to photograph them as much. It’s true that in a lot of photos (including good ones) currently one can’t actually see any crew. However, you know that they’re there!
I had never thought about this until this moment; but for whatever reason, I don’t think I’d shoot trains as much if they were robots.
Euclid: Agree in part but would point out two things. 1. Hazards such as stalled autos or trucks on crossings occur and stopping or at least slowing can help avoicld disastrous derailments. 2. Automated freight trains will probably be much shorter** so stopping distance is reduced.
** Shorter trains on tighter headways should mean better service to more end customers.
The short answer for elaborate sensor fusion and interlocking AI/ES is, in a word, lawyers. There are a raft of additional reasons, but that one alone is sufficient.
That the railroad applications will benefit from costed-down or OTS ‘vehicle technology’ should go without saying, but there, I’ve said it again. Many enabling technologies that would be ridiculously expensive and brittle if developed, say, like NAJPTC could be remarkably good if systems-integrated from areas where ‘early adoption’ is subsidized in some way.
At first, it will be the novelty. After that, I have my doubts that the railroads will go to any special lengths (ie, tribute locos, etc) to attract the fans.
You may be right. I don’t remember his name, but a rail writer back in the late 50’s said something to the effect of “When all the steam locomotives are gone trains will be no more interesting than a conveyor belt.”
Obviously that didn’t happen, but he may have been just a bit premature. Robot trains? Can’t say that idea excites me too much.
The only positive thing I can think of in favor of robot trains is there won’t be any more head-end crew traumas due to tresspasser strikes. A robot won’t care.
For that to worry the railroads, they would have to believe that lawyers can make the case that an engineer could have stopped a train in time to avoid a collision whereas the automatic system is incapable of that. It would easy for the defense to prove that neither an engineer nor the automatic system could stop a train in time to avoid a collision in many cases.
I am not saying that the automatic system does not need any scanning detection for danger ahead. I am saying that what is being proposed by the FRA is sensor overkill.
The main issue, as Charlie mentioned, is spotting stalled vehicles on grade crossings. That is the one contingency that needs the intelligence and quick reaction, no matter whether artificial or of a human engineer.
I recall reading that Rio Tinto automatic trains are equipped with a grade crossing sensor to spot obstructions on the crossing and apply braking if necessary. So that would also be needed for U.S. automatic trains.
But instead, the FRA is proposing a sensor “platform” featuring the following equipment:
VISUAL CAMERAS
THERMAL CAMERAS
INFRARED CAMERAS
I got the impression that they were testing a variety of sensors to see which best suited the need. Not that they intended for all of them to be used.
Several of the sensors are similar in function.
From the FRA report linked in the original post:
“To satisfy SP [sensor platform] requirements, TTCI concluded that a suite of various sensor types, working in unison, will be necessary.”
“Field testing must also explore the use of multiple types of sensors and the fusion of data from those sensors to enhance SP functionality beyond what can be done with a single sensor type.”
While they do not say how many sensor types they will use, or how many of each type, the thrust of the document is the assumption that sensing must be able to thoroughly see and assimilate as much detail of the forward field as possible. The obvious reason for this assumption is that the content of the forward field will need to be thoroughly assimilated by the automatic operating system in order for it to drive the train.
In other words, it is the same as the human engineer sitting in the seat and watching every detail ahead and factoring them as needed to make the decisions about operating the manual controls. Therefore the so-called, Sensor Platform is providing the same sense data for the automatic operator as an engineer’s eyes provide to the human engineer operator.
That is the way it is for road vehicles where there are many decisions to be made by a
The visual sensing capabilities (IR, heat, UV and glare filter, telescopic) and reaction times would be far better than a human operator.
I find it a little sad that a fifth of the way into the 21st Century there are still people who think a train can be always, or even primarily, brought to a stop short of a vehicle stopped on a crossing by use of sensors only on that train’s locomotive, filling in for the ‘eagle eye’ of an engine crewman.
Where sensor fusion and intelligent sensing is most valuable is in the modern version of restricted speed, where the higher net resolution and lower latency will vastly improve safety.
One concept I’ve been fond of is the idea of extended ‘haptic space’ – for a train this would include determination and if necessary filling-in and confirmation from things like wayside cameras, scanners or loops, or from strategic orbiting or launched drone coverage. In this context any vehicle or other intrusion that is tracked by a ‘third party’ for enforcement can be relayed (ideally, autonomically with high reliability and low latency) to trains well in advance of their brake limits; even false positives result in not stopping, but the ‘right’ counterpart of restricted speed on approach.
Can’t forget rock slides, large trees, MOW vehicles, etc.
Sensors fixed along railroad corridor versus sensors on locomotives:
Declaring that autonomous running must include 100% of line advanced hazard detection places an unnecessary burden on the challenge of autonomous running.
I think slide protection will just continue as electro-mechanical slide fences linked to PTC. Generally, there should be a distinction between locomotive-borne sensors and fixed sensors along the track. Both make sense but their purpose differs. Fixed sensors could be maximized to include any possible track damage or obstruction such as broken rails, collapsed bridges, landslides, rock falls, washouts, and all forms of encroaching vehicles either on the rails or ground. And most importantly, they would give warning as soon as the obstruction occurred rather than when it comes into the line of sight from the locomotive. That would allow time to get the train stopped in many cases.
But again, I mention that there is a distinction between sensors detecting random dangers versus sensors to know the train location and operate it accordingly. The automation does indeed need the latter, including sensors and programing to replace the capability of the human engineer. But even with that function, sensors are mostly needed for reacting to grade crossings if there is time to stop or slow down enough to make a collision non-fatal, or possibly stop in time to avoid a collision. The rest of the engineer’s function would be addressed with the program and PTC.