A global archive of independent reviews of everything happening from the beginning of the millennium
Read our Copyright Notice click here
For publication dates click here
Video briefly explaining sensor fusion and object recognition.
For two years I've been gently annoyed by a question that is posed as if it were a philosophical dilemma or ethical choice, especially because you hear it so often: what will a driverless car if do it has the choice of ploughing into a group of schoolchildren or driving into a wall killing its innocent occupants?
The answer almost certainly is that the onboard computers will never have to make ethical choices of this kind.
If the car recognizes an obstacle that it must avoid - children instead of a balloon, say (and this is why good object recognition is essential and most projects do not yet have it) - the car will apply the brakes, hard if necessary, and probably much earlier than a human could, to slow down. If there is a gap to go through to avoid the children it will go through it, if not it will continue to deliver maximum or optimum deceleration, with the result that the children and, maybe, the occupants (they should have airbags) may suffer injuries. What the algorithms will not deliver is a 'car commits suicide' option where it deliberately drives into a wall or something else.
On this website I once raised the question of what a driverless car would do if a large bird swooped down apparently intent on hitting it - would it brake, possibly occasioning another car to run into the back of it? What appeared to be the answer from Google came back indirectly: the car will ignore birds.
So a rarer 'ethical' question might be more pertinent than the common one: If a large piece of masonry was coming down off a building site 20 metres ahead, perhaps too close to allow much deceleration, would it drive at it or swerve and hit a wall to avoid it?
It must be said that Google is a company that has the resources to answer this one but I fear a lot of autonomous vehicle projects do not and so I am sceptical of most safety claims for autonomous vehicles in a way I was not when I was a pioneer researching how they could be introduced into the U.K. three years ago.
For many projects the likely answer would be that the object would not be recognised by the computers and if picked up by the vision systems ignored like the birds.
There is also a fair amount of nonsense talked about who will be legally to blame for accidents but a dilemma of this nature can already be presented by today's technology.
If you have a car with infra-red sensors that can see an animal crossing the road in the dark before the driver can and the car applies the brakes hard, without the intervention of the driver, with the result that another vehicle runs into the back of the car, who is to blame?
An AI-focussed, venture capital funded driverless car company, Five AI, has launched in Cambridge. It is probably not going to end up owning many driverless cars of its own - it is not a vehicle manufacturer by any means - but it is a good bet to develop AI that can be used by future driverless projects in an urban context and so this company looks like one that could eventually sell for a healthy multiple of initial funding.
15 September 2017
The Kangaroo Konundrum
Purely algorithmic decision-tree object recognition cannot reliably identify a kangaroo crossing the road. Deep learning artificial intelligence should be able to do so and then cloud based parallel processing should be able to verify the fellow is a kangaroo in real time. When you have a non-sentient system that does this, come back to us and we will set you the next puzzle.
21 February 2018
The Kangaroo Konudrum is now easier to solve.
Last week Google announced it would make use of its proprietary tensor processor unit chips, specifically designed for artificial intelligence tasks, available to third parties via its cloud computing service, focusing on computer vision technology, which can train computers to recognize objects.
All that is needed now to train the computers is lots of imagery of kangaroos bouncing across roads!
Would-be gatekeepers of object recognition technology are being sidelined in effect.
In the context of future autonomous navigation on roads, that is important. There are so many object-related problems to solve.
If, for instance, I should want future driverless vehicles to be able to recognize scaffolding around buildings which was not on prior maps, I would need to have an object recognition neural network trained to do so.
A driverless vehicle on the move would need to recognize that a building that was on a map had been occluded, identify the occlusions as scaffolding and use decision tree algorithms to make decisions about its robotic functions, like driving.
A problem is do sufficient labelled images - RGB, lidar point cloud, radar and infrared - exist to train a neural network? Once multiple lidar equipped vehicles are travelling around to acquire them, that point could be reached.
Can any semi-autonomous vehicle at present recognize scaffold poles standing in the roadway with clear air around them?
Technically a driverless vehicle is classified as a robot but the object recognition functions fall squarely within the domain of artificial intelligence. Motorised autonomous vehicles on public roads unable to recognize objects are non-starters.
In reality you have to informally outsource the problem solving to the wider market as there are many more challenges and problems than any single company can manage.
23 February 2018
The making available of Google TPUs to third parties is one of the most significant events in digital history.
There will be an explosion in artificial intelligence work as a result.
(It is akin to Amazon's historic decision to rent out its cloud servers. By renting out, the system will be honed, in effect outsourcing some problem solving to customers.)
Although the early work on driverless cars was done by robotics enthusiasts, developments in robotics have taken place at a slower rate than those in artificial intelligence.
Google bought Deep Mind and sold on Boston Dynamics.
Now the route to autonomous vehicles is via artificial intelligence and machine learning. The skills needed in this area are in very short supply given competing claims on their use.
12 March 2018
Emphasis as to what is important differs depending on which areas a corporation's products have strengths in.
Here are two useful podcasts from Qualcomm from last year on artificial intelligence:  .
12 March 2019
I was working for London Underground when the Docklands Light Railway first went live. That is an example of automatic rather than autonomous transport vehicles but they remain within rigorously defined physical bounds. I have been asked, in the City, why we cannot have automatic rather than autonomous car transport. The answer is that we can but nothing much has changed - it must stay within rigorously defined physical bounds.
Autonomous vehicles will not, in all events, be able to have all their capability onboard. Beyond certain parameters they will need to have two-way communication with the cloud.
Latency matters in relation to this communication. The Qualcomm Snapdragon X55 5G modem, one of the first of the new generation of modems, or its successors, might be used to connect future autonomous vehicles but the question is will the 5G networks ever be dense enough to allow the vehicles to go beyond cities and motorways?
Their autonomous capabilities would let them travel between widely spaced data waypoints but 'connected automatic vehicles' or 'thin client vehicles' would be much more circumscribed in where they could go.
It would be technically possible to equip every truck that crossed a border with one of these modems, which would communicate with electronic border posts automatically, transmitting the manifest of what was onboard.
22 March 2018
The latest fantasy we have been spun is that driverless truck deliveries are on the way to Britain. The unions need not worry. I like the song Luxury Liner but 40 tons of steel, or 44 tonnes of juggernaut as you get in Britain, gone haywire or with malware fed into the guidance system because discreet frequencies have not been used for the guidance communications will be able to go straight through the base of a skyscraper killing or injuring the pedestrians and office workers in the way and taking out support columns, too. Besides, there is not a guidance system in the world in existence that can navigate a truck through winding city centre roads and no medium term prospect of one let alone of an autonomous truck that can deliver to your local city centre supermarket.
No, if you want laden 40 ton trucks travelling autonomously, stick to closed mining roads with no pedestrians or other civilian traffic on them in Australia or elsewhere and do not expect the guidance systems to recognize kangaroos crossing.
As on a racetrack, object recognition will not be the insuperable problem because there will be next to no objects to recognize.
It is time to rein in techno-fantasies fast.
Today is the anniverary of when a hostile vehicle ploughed into pedestrians on Westminster Bridge killing four people metres away from Parliament.
Ideally, all testing of autonomous and semi-autonomous trucks and vans should be banned in London for the forseeable future. The guidance technology is absent.
Cars are lighter. The only successful trial in Britain of a semi-autonomous vehicle with some capabilities approaching those of an autonomous vehicle was a year ago, on 28 February 2017, by a Nissan vehicle with an Evening Standard reporter onboard near the Excel Centre in the early hours of the morning where many of the roads are private and there was little other traffic.
12 April 2018
Video (source BMW).
As of today, this is the most sophisticated you get in self-parking.
26 August 2018
This situation is innocuous but how would an autonomous vehicle interpret it?
The challenges first of vision then of interpretation are so large that artificial intelligence in its current state of development cannot deliver safer solutions than the experienced driver in the majority of situations.
The premises on which autonomous vehicle programmes are based therefore need to be reconsidered.
30 August 2018
Meanwhile advanced semi-autonomous vehicles can stick to exceptionally well-mapped routes, protected routes and those with white lines.
For a country with a genuinely small land area like the U.K., part of the solution could be to put a low level satellite over the top, so we should invest in satellites come what may, but the incapacity of algorithms to make consistently correct decisions from some kinds of data is the stumbling block.
6 July 2020
The U.K. government's proposed purchase of a stake in a company with a constellation of low orbit satellites is a remarkably useful development for future autonomous lightweight vehicles as it provides a means of externally assisting navigation with two way communication and low latency.
Combined with our own means of verifying location, vehicles could navigate without conventional road maps or GPS, or, when the first of these is absent or the second is switched off.
Reviewed by ANDRE BEAUMONT
6 February 2016
The news that Britain will be experimenting further, on a very limited scale, with removing white lane markings from the middle of roads leaves one in two minds. This may slow down motorised vehicles so reducing accidents but it does not help pedestrians.
In many of those places with no road markings, motorbicycles, buses and cycles arriving on the road space pedestrians are using or crossing is a nightmare, because they have little in the way of visual cues to know where they will be or how they are arriving, and it will be worse for those with impaired hearing or vision.
For example, if the pedestrian is crossing a road and is caught out by a rapidly turning vehicle he can stand on the white line and the driver will understand what he is about but it is harder to see, hear and anticipate everything on an unmarked road.
Like many cars now have, human beings have their own 'sensors' - eyes and ears - and they use them intensively but they need to be given cues that they can read. Which brings us tangentially to future autonomous vehicles.
A Mercedes truck that partially uses autonomous guidance (this one, at the entry for 12 May 2015) is heavily reliant on lane markings to be 'autonomous'. In many ways it is not truly autonomous partly because it will not drive in autonomous mode off the type of highway that has good markings.
Mercedes' research is near the top of the tree for autonomous vehicles, especially in dollars committed.
A lot of other 'driverless car' demos, though, are little more than the combination of lane keeping technology, cruise control and some less than fail-safe processing of sensor data. The vehicles are also in no real sense driverless.
Here at Worldreviews we are advocates of driverless cars but ultimately we are not starry eyed.
Some driverless car projects will come onstream for restricted public use in 2017-2020 but they will be in geographically limited areas that will have been mapped in advance more intensively - by orders of magnitude - than urban roadscapes ever have been before.
Some projects will have good sensor fusion, some good object recognition - but most probably will not have in the early years.
What they will rely on to operate, though, is lots of lane markings. These are the objects they need most of all to recognize.
A pedestrian having to second guess a bus on a road with no lane markings is bad enough, a driverless white van on the same road quite a harder prospect. How would you ever guess its intention to stop in a particular place or, worse, mount the kerb? (Driverless delivery vans are probably the least feasible outcome in the near term.)
For pods and shuttles, vehicles currently designed to move through shared space at low speed, and which we should not confuse with driverless cars, there is the prospect of navigation that is not reliant on lane markings.
These would include the recently announced NVIDIA-powered WEpod of Delft Technical University that uses some of the GPU manufacturer's deep learning capabilities and also a promising project in Greenwich, UK which looks like it is run by a commercially competent consortium which includes the Transport Research Laboratory and Heathrow.
The latter project has 3D local mapping provided by Oxbotica and is partly funded by the EU.
The Dutch WEpod is an iteration of the EZ10 vehicle, supplied by EasyMile, and part of the EU-funded CityMobil2 project. Other EZ10 trials are being run or rolled out in a handful of European countries. The vehicle is electric, semi-autonomous and, at present, is low speed and concentrates on 'last mile' routes that are protected from other vehicles. Many of the routes have the potential to terminate at railway stations.
[Where these vehicle can run has proved to be circumscribed.]
It is reported that Sheik Mohammed bin Rashid Al Maktoum would like to see a quarter of all road journeys in Dubai undertaken in driverless vehicles by 2030.
An EZ10 vehicle, which as its name implies carries ten passengers, is already being tested in the state. It does not require lane markings but vehicles with a lot more artificial intelligence, like Google's cars, will. For these, the lane markings issue need not be a problem, initially, in Dubai. Some of the roads have twelve lanes so dedicating some to driverless vehicles should not be a problem. Snow obscuring the markings is unlikely to be a problem; sand, perhaps.
For those of us who have been studying driverless vehicles and their constituent technologies from the outset, one of the main problems is finding conceptual realism in those we talk to. In 2016, the conceptual frameworks for them require an understanding of many disciplines all at once, from politics to sensor fusion.
A single discipline, like computer coding, cannot deliver the majority of the answers and neither can a turnkey contractor. Conceptually, the rendering of what is being mapped may be one of the most advanced disciplines.
Consultancy reports from the financial world extrapolate safety projections and financial benefits from minimal practical experience so these have to be taken with a pinch of salt.
A quarter of road journeys is about right as is the target date of 2030. There is no way that driverless vehicles can supplant legacy vehicles, except progressively, and never absolutely. Are we to have unmanned JCBs and fire engines on public roads?
Vision and realism are required. From decades of observing Sheik Mohammed's equine operation in Newmarket, I would say he has it.
I had a brief conversation with the Director General of the EU responsible for autonomous and connected vehicles in May and watched a presentation by him which contained nothing inconsistent with the direction of travel the U.K. should go in but where will Britain be in January 2019, possibly the date Brexit takes effect?
Coordinated with Europe on connected vehicles and road security standards and probably behind the curve on autonomous vehicles unless Volvo or another external player can bring a successful trial to Britain.
The U.K. autonomous vehicle effort took a diversionary wrong turning at the beginning of 2015 which took it down a cul-de-sac.
Capabilities to read lane markings, for instance, were not in evidence.
Fortunately, not all conceptual and technical expertise went down this road and a shift in effort is now taking place which will re-centre efforts around the U.K.'s known and innovative capabilities in vehicle testing.
It is apparent that for autonomous vehicles and semi-autonomous vehicles to read and follow lane markings with a high degree of accuracy they will have to rely on machine vision systems. These will be able to see in the dark (or the best will) but their effectiveness will be degraded when lane markings are wet. When they are obscured by snow, sand or proximate vehicles, distance measuring sensors - lidar, radar and ultrasound - will have to fill the gap in locating the vehicles relative to their environment - requiring strong sensor fusion capabilities.
Until this package is finely honed and road markings upgraded, limitations remain on autonomous commercial vehicle testing and on semi-autonomous commercial vehicle platoon formation testing.
Some of the systems found on the experimental Mercedes truck above can now be found in Mercedes cars, like the new E-Class, such as autonomous gearchanging in advance of junctions based on GPS positioning. BMW also has an impressive suite of driver aids on its new 5 Series. The cars, however, stay firmly within the realm of the semi-autonomous at most.
The E-Class, for instance, requires the driver to occasionally touch the wheel when not actively steering.
The goal of fully autonomous navigation in all driving conditions, primarily pursued by Silicon Valley companies, is a long way off being attained, though on protected, defined and heavily mapped sets of routes, cut down versions of autonomy might be demonstrated this year.
It will be a long slog to get all the requisite technologies developed and to mesh.
It is not like developing an app for remotely operating a home central heating system. In fact, it is a harder challenge than Nasa returning to the moon. You are not just travelling in air and a vacuum but in a dynamically unpredictable environment.
Minimum investment for serious players runs into hundreds of millions of dollars just to develop and partially prove concept. Thereafter autonomous technologies will require billions in infrastructure investment per major city if the cars are to be used on the usual roads with traffic lights, complex junctions and roundabouts.
In the early years we could be dreamy about driverless cars and believe the hype but now that is impossible.
Take the example in the side column for the entry for 21 February 2018.
If there are not enough labelled images of scaffolding to provide training data for deep learning neural networks to begin to attempt to identify scaffolding and any alternative using another form of machine learning, like reinforcement learning, not at a sufficiently advanced stage of development how can a claim that driverless cars are nearly ready to be rolled out for city use be supported?
If the scaffold poles are in the roadway supporting something above but unenclosed, which type of sensor will spot them?
If they are enclosed in plastic sheeting, RGB imaging might result in the flagging up of a building enclosure without seeing the poles behind.
Radar imaging might show the poles but not the enclosure.
Which companies have vehicles that have good sensor fusion of RGB and radar images whilst on the move?
How can claims that driverless cars will be orders of magnitude safer than human-piloted vehicles be supported?
If the identification of scaffold poles is not a priority for the artificial intelligence community how costly is it to protect the routes used by autonomous vehicles in the meantime?
Must all routes be protected from building enclosures?
Must a route be delineated by a continuous concrete barrier one side of which a scaffold pole may not be placed?
How exactly is an autonomous taxi service in the city meant to be realistic until countless researchers and companies have broken down the problems using cloud based artificial intelligence?
Is certifiable trustworthiness a reasonable expectation of current AI?
With an arbitrator's hat on, and a standardizer's, too, being a bit of a polymath, I have long argued that legal disputes relating to autonomous vehicles cannot be resolved using traditional legal procedure. A hybrid, part technical, part human judgement will have to take its place.
At its simplest, data logging will have to determine whether anyone sitting in the driver's seat touched the controls prior to an accident but it will go well beyond this.
The presumption that if someone runs into the back of you they are at fault that often now applies might have to be reversed. An autonomous vehicle ahead might not have proceeded as any reasonable following human driver might have expected.
Where deep learning is applied to object recognition it may not be possible to generate the transparency of decision making necessary to certify trustworthiness.
Some may game the system, like pulling in front of an autonomous vehicle to force it to brake.
Others may inject malware into the data stream so that adversarial examples confuse the AI.
This goes beyond academic disciplines to highly pragmatic ones.
Initially, where AI has made decisions that impinge on fairness there should be an appeal to a human being or a panel of three people, possibly post the event.
So, hypothetically, if AI has decided that someone should not pass their driving test, or lose their driving licence, based on sensor data, a party questioning the fairness might have a chance to resit the test with just a human examiner in the former case or have a panel of three people review the sensor data and all other evidence in the latter case. Currently, judges and magistrates may or may not be qualified to review sensor data.
6 February 2019
Nissan Leaf based semi-autonomous vehicles have been trialled near Abingdon and their robotics are likely to be quite good and scaleable in other applications but it is not robotics that are central to driverless cars.
The Nissan Leaf is clearly the best platform built in Britain for autonomous vehicles. Beyond the short term, an all electric vehicle is essential and the rival electric pod project did not address important issues. Internal combustion for semi-autonomous vehicles is all well and good but only for the development stages and internal combustion road vehicles will always need a safety driver.
We have a different conception of how driverless cars should position fix on the road and in relation to other road users from any model used in Britain.
16 May 2020
I remember in the financial crisis of 2008 saying to a banker who tried to con me into put a sum on deposit without a receipt or stamped counterfoil of any kind : "We are solvent; you are not." Instead of denying it as she should have done she cringed heavily and said as if it taking her word was all that a customer should do : "Oh, so you really want a receipt?"
So I knew it was true. Within days a massive hole of multiple billions opened up in the bank's balance sheet and it sought government assistance.
The money went into a current account for which the receipt had to be insisted on.
For some years our view about autonomous vehicles has been similar: "We know how to assist autonomous vehicles in their navigation; you do not."
That is fine. That bit we have not imparted to anybody.
There is before coronavirus and after coronavirus. The biggest change is in the type of vehicles that should now emerge with a modicum of autonomous assistance.
If a government cannot even write the code for a contact tracing app without multiple flaws let us not assume there will be vehicles that can navigate around safely and take your instructions except within narrowly defined parameters.
17 May 2020
Our tiny consultancy had to do the thinking on how driverless cars would co-exist with the legacy vehicle stock for up to 30 years. Now we know the answer to that one: there will not be driverless cars that replace it.
The thinking was valuable though. In the midst of a Covid-19 lockdown many cities are very pleasant places with scarcely a car moving.
What is now needed is a way that very much lighter four, three and two wheel electric vehicles can replace and co-exist with the heavy, fast and polluting vehicles on the roads.
The environmental warriors have played it wrong to date. They have refused to take the low hanging fruit first and have gone for all or nothing. The result is that they have been offered next to nothing: an end to diesel and petrol vehicles in law by 2050 or 2040 or 2035 meaning nothing much happens till then.
Those mayors that have introduced no diesel zones or no diesel days have been much more politically able. Diesel and petrol are not equal. One reduces life expectancy much faster than the other, on a scale similar to Covid-19.
We have seen it before.
The bien pensants want others to retrofit domestic houses with insulation [polyethelene-cored or foam insulation in one's home - no thank you at any cost] whilst wilfully ignoring schools that leave their lights on all night and municipal buildings that leave air conditioning and heating blaring through the lockdown whilst not taking on government and commercial property owners at all as they are more powerful.
Early projects in Germany, explained to an audience at Cambridge University nearly a decade ago, showed how devilishly difficult domestic retrofit is at a practical level, how it is beyond the means or space availability of most residents and how a lot of heat loss is through party walls to neighbours.
Formula 1 technology shows that you can get a lot of horsepower out of a small petrol engine and even more out of energy recovering hybrids.
The product cycle of a car is about seven years. So we must say that there should be no new diesel passenger vehicles from 2027 and from then petrol engines should be no more than 750cc capacity.
It is no use European governments saying that behind the scenes they encouraged investment in diesel technology and so they need to protect the investment of the car manufacturers. That is not the business of government. The market will supply what there is demand for and reward the investment of those that correctly meet demand.
Japan is already there. There are few diesel cars, a segment that is composed of low capacity, low weight petrol cars and a flourishing line in hybrid cars.
14 November 2020
Boris Johnson is doing very well on this. An announcement is awaited for later today.
No new combustion engined cars after 2030 with an exemption for hybrid cars until 3035.
The market will adapt with new classes of vehicles. It gives Britain an opportunity to be a leader in the new classes as tooling up will have to take place anyway.
Vehicle manufacturers of advanced off-road capability vehicles might opt for a diesel hybrid for a little extra torque but mainstream car and innovative vehicle manufacturers will either opt for all electric or petrol-electric hybrid. No wants to put a heavy second engine in an electric car and so the petrol engine component will be low capacity and that is all that is needed to charge the battery away from charging points and to augment performance further when required.
Pollution will fall, diesel will be relegated to mainly commercial use cases, carbon targets will be hit earlier and significant savings will be delivered to healthcare systems.
18 November 2020
Press briefings last night confirm a similar outline as now being government policy.
The exemption for hybrid vehicles and the early phasing out of new diesel cars were advocated first here. It was always probable that petrol cars would join them.
Some commercial use cases have already gone electric. Going green is a piece of cake even if you start blue
Designing cars? Put the world straight first. Cut pollution!
27 December 2020
As Britain no longer makes cars in the very small car segment there are few impediments to innovate in it after the end of the transition period following Brexit.