Autonomous driving, even in the scenarios where a robust probabilistic function or a model of the behaviour of obstacles, or that of agents representing the obstacles, is available, along with strict (mathematical-) projection of, or (in mathematical functional form) superposition of, strict traffic rules, remains an active area of research, both in terms of theoretical and experimental research paradigms.
Current motion planning and decision making research, globally, in the autonomous driving industry, at large, focuses on developing the ability to negotiate obstacles on broad roads, with strict traffic-rules, along with assumption of availability (or computational ability) of the probabilistic functions encoding the behaviour of the agents representing the obstacles. Negotiation of bi-directional traffic on a single lane road, where the autonomous vehicle (or Ego-Agent) might be required to shift off the road, and drive through an uneven patch of terrain to facilitate successful negotiation, remains an uncharted terrain for the industry at large.
Such abilities have not been demonstrated by any autonomous driving technology startup as of writing of this document.
Our earlier research in enabling autonomous navigation through tight-stochastic-dynamic-adversarial environments led to the development of an algorithmic framework, an end-to-end holistic framework, that used multiple reinforcement learning (RL) agents to perform following tasks:
- Understand the intent of other obstacles in the environment.
- Compute the motion and behavioural commands for the autonomous vehicle to navigate through very tight traffic scenarios.
This algorithmic framework was demoed on November 15, 2017, when Swaayatt Robots was a one-person (Sanjeev Sharma) startup.
The algorithmic framework had tremendous capabilities to negotiate tight-stochastic-adversarial-dynamic environments as can be seen in the above video. This framework was specifically developed to deal with adversarial nature of the opposing agents in the environments. For example, in India, in the absence of strict adherence to the traffic rules, agents, representing two wheeled vehicles, are not only stochastic but also adversarial in nature. It is the task of the ego-agent to ensure collision-avoidance. In case of an accident, Indian law favours the smaller vehicle. Thus, there is adversity in the traffic-dynamics on Indian roads.
One might argue, does a solution to this problem have a practical significance outside India? The answer to this question, while debatable to an extent, holds the key to solving the Level-5 autonomous driving problem. Only if we ensure safe navigation in such traffic and environmental conditions, we can really be sure of the safety of such vehicles elsewhere, where the traffic dynamics is simpler, and environments are significantly structured.
While Sanjeev was developing this framework, he asked himself a very simple question; if I were to drive in this colony road, would I stop for every obstacle coming from the opposite direction and yield to them? Or would I use my driving experience and judgement and negotiate without completely yielding. In India we do this on a regular basis. Furthermore, since the beginning of his autonomous navigation research, he focused on enabling autonomous driving in world’s most difficult traffic-dynamics imaginable for autonomous vehicles, along with enabling navigation in completely unknown environments, i.e., without the high-definition maps, just like humans do.
Formulating this problem, this biological inspiration, mathematically required him to work at the intersection of convex optimization, reinforcement learning, heuristic search, apprenticeship learning, mathematical topology, and deep learning to develop this framework in 2017 and demo in our autonomous vehicle. While this was a very robust framework, in the absence of any funding back then, this pioneering work couldn’t be scaled further at that time. This was a first ever successful demo of reinforcement learning for practical real-world autonomous driving. Even more so, this was a multi-RL agent demo, for learning a behavioural capability that was unimaginable in the context of autonomous driving back then.
Present Work
Recently we began addressing this problem of bidirectional negotiation again, at a very large scale, to solve the Level-5 autonomous driving problem.
Whether autonomous vehicles should be allowed to execute such a behaviour can be topic for a debate. However, in India, many of the roads single lane road, where only one vehicle can fit at a time, and if another 4-wheeler comes from the other end, then both the vehicles will have to shift to the side of the road to allow each other to pass-through. In case of a truck, the deviation from main road is even more significant, as can be seen in our demo. Furthermore, solving this problem requires much higher level of intelligence that what is demonstrated by the contemporary autonomous driving technology demonstrated by North American and European companies. Usually autonomous vehicles come to a complete halt, unable to compute action to allow negotiation on broader roads, in the presence of opposing or laterally crossing obstacles, let alone negotiating bidirectionally on a single lane road. Humans on the other hand, are quite adept at such bidirectional negotiation, like drivers in India, even at high-speeds.
Taking this biological inspiration, very recently we started development of a motion planning and decision making algorithmic framework, present from a classic standpoint, i.e., without using deep learning, in the present work. Very recently we demonstrated this algorithmic framework in our off-road autonomous driving demo in September 2023, where it enabled negotiating very tight spaces, at low speeds, on a single lane road.
The framework allowed our autonomous vehicle to drive off-roads, on-roads, as well as negotiate tight obstacles, such as avoiding a tractor or a very narrow curved region. Furthermore, in the situation around 30-seconds in the video below, it allowed our vehicle to negotiate a tight passage through a vehicle parked on the left, and a bike approaching from the front on the right, successfully, without bringing the vehicle to a complete halt.
Bidirectional Negotiation on Single Lane Road
Having successfully negotiated tight obstacles on narrow on- and off-roads, we started scaling our framework, which has multiple agents (not necessarily RL agents) acting over multiple horizons and a custom built motion planner to compute safe motion and behaviour commands to allow autonomous vehicles to negotiate bidirectional traffic on a single lane road.
The navigation task over here is extremely challenging for multiple reasons, where some of the most prominent of those are listed below:
-
- Not knowing how much to yield for an incoming obstacle. This is a multi-agent negotiation problem. When viewed from a Game Theoretic perspective, one can clearly see that ego-vehicle’s action will affect the incoming vehicle’s action and behaviour, as they are being driven by human drivers.
- Human drivers, dictating the obstacles’ behaviour, are proficient at close avoidance at high-speeds, and can choose not to slow down at all, or not to yield at all. Whereas, an autonomous vehicle will typically go for a slow-down and then avoid.
- Not knowing which side to avoid from with absolute mathematical certainty. Although there is a very high probability that the incoming vehicle will at least follow the traffic-norm of keeping on the left (our right) when approaching our vehicle. However, this very assumption, if at all assumed in our model (this is a highly confidential work), was also violated at around 5 min 21 sec into the navigation, as seen in the video below, where one vehicle was a bit left off-set on the road, and the vehicle behind it was a bit right off-set on the road, but both of them chose to avoid from (our-) left (their right — violating the traffic norm on Indian roads).
- Not knowing how much to yield for an incoming obstacle. This is a multi-agent negotiation problem. When viewed from a Game Theoretic perspective, one can clearly see that ego-vehicle’s action will affect the incoming vehicle’s action and behaviour, as they are being driven by human drivers.
The bidirectional negotiation has many other challenges, and this article refrains to discuss them for confidentiality and for brevity. All of these combined make such a negotiation extremely challenging. While there are a significant number of improvements possible in our framework, we are working on them as of now, including developing a deep neural networks based non-holistic, and later holistic, framework, which we will showcase in January 2024. The present demonstration below showcases our very deep Level-5 autonomous driving research, and tangible technological outcomes from this research, to make autonomous vehicles significantly much more capable in negotiating tight stochastic dynamic adversarial obstacles, just like the human drivers do.
One can notice that at around 25 seconds into the navigation, a bike, and a large truck behind it, appear in the vehicle’s field of view. Our vehicle was executing a turn at high-speed, and had to slow down, and shift off-roads for both the bike and the truck to successfully execute the negotiation on this very tight-curved part of the road. Our vehicle’s more than 50% part was off-roads, whereas the truck did yield, but not to that extent.
At around 5:25 our vehicle was confused as both the incoming vehicles occupied almost the entire road, and after the first vehicle almost crossed, although from the wrong side, our vehicle anticipated that the other vehicle will follow the driving norm and will try to avoid from (our-) right. Thus, the vehicle turned its steering to left, and when the other vehicle showed a tendency to cross us from (our-) left, the vehicle stood and made its steering straight, as there was nothing to be done in this case but to wait for the other vehicle to pass.
At around 6:45 into the autonomous navigation, we can see that a Tata-Safari vehicle stood still on the road, and decided not to yield at all, until our vehicle already started the avoidance manoeuvre.
This thus became a generic (nearly-) static obstacle task for our vehicle, which our autonomous driving technology has become an expert at.
Towards the end of the video, one can see the speed at which the vehicle crosses the very initial curved portion of the road, and the manner in which it navigates in the absence of any obstacle in its path, vs the behaviour and the level of departure it decided to go for in the presence of a truck in its path.
Future Work
Earlier in 2017 we demonstrated an end-to-end holistic framework capable of negotiating very tight regions using only two cameras mounted on the front bumper of the vehicle, computing directly the control commands from the input images. This work on the other hand was a classical motion planning and decision making algorithmic framework developed to handle such sophisticated navigation and negotiation tasks at a very large scale. Presently we are further increasing the capabilities of this framework, along with developing deeper variant of this, which will solve Level-5 Bidirectional Negotiation Problem, first in a non-holistic, and then in a holistic manner — taking inspiration from the November 2017 work — mathematically modelling how humans drive.
We are researching on several avenues for this framework, and for solving generalized autonomous driving problem, including mathematically modelling the human driving behaviour to develop the Level-5 autonomous driving technology.
