This demos showcases our autonomous vehicle at Swaayatt Robots negotiating traffic-cones (obstacles) at very aggressive speeds. The location of the marked cones was not known to the planner beforehand, and thus the motion (time parametrized trajectory computation with obstacles avoidance) planning and decision making was done completely on-the-fly . Furthermore, to make the scenario more challenging, and to test the robust-aggressive behavior policy, the agents in the framework were constrained to not act on any obstacles unless they are within 24 m radius. Typically, an autonomous vehicle should be able to react quickly to overtake, or to avoid, any sudden unforeseeable obstacle or pedestrian on the road to avoid fatalities. This motion planning and decision making algorithmic framework adds to the overall intelligence, allowing autonomous vehicles to avoid fatalities on the road, to achieve Level-5 autonomy. We demonstrated two different obstacles avoidance patterns: 1) Zig-Zag 2) Obstacles on the left lane For the latter, the system was constrained to bring back the vehicle to the left lane at aggressive speeds. One can notice that our motion planning and decision making framework effortlessly avoided the obstacles. Given the vehicle platform, Bolero, which has a very high body-roll, a human driver would face difficulties when driving beyond 40 KMPH in such scenarios. During the zig-zag motion, the vehicle consistently kept speeds above 45 KMPH, slowing down to 39 KMPH only at one instance. During the aggressive planning phase, the vehicle consistently kept speeds above 50 KMPH, slowing down to a minimum of 47 KMPH. The entire algorithmic framework with 5 classical (with one reinforcementlearning-) agents , runs at 800+ Hz on a regular i7 processor, single thread. The entire algorithmic framework includes an inherent trajectory tracking algorithm, which generates adaptive commands to the pure-pursuit algorithm to compute the steering angle. This entire operation requires only one single hyper-parameter, controlling the aggressive behavior of the framework.

In this demo we showcase autonomous driving off-roads at night, with obstacles avoidance at high-speeds, paving the way for fully autonomous vehicles, both on- and off-roads.

This was our 9th demo off-roads, where earlier last year, in Feb 2024, we showcased the ability to negotiate bidirectional dynamic traffic off-roads.

Going forward this year, we will be showcasing our reinforcement learning (RL) first approach towards solving the problem of off-roads autonomous driving.

Off-roads autonomy is a tough problem to solve, due to non-availability of HD maps, and also due to supervised deep learning approach for solving the perception problem, which works well for many of the on-road frontiers, being infeasible from a resources point of view. Modern on-road autonomy stacks make use of HD maps to also embed the motion primitives and to compute avoidance maneuvers on-the-fly, which is not possible off-roads.

We are thus taking an (inverse-) RL approach towards solving the off-roads autonomous driving problem, where the autonomous agents inherently learn a representation for decision-making under uncertainty and motion planning, without requiring an explicit (projected-) representation of the world, as may be computed by different localization and perception modules.

Autonomous driving and obstacles avoidance at drift speeds, challenging the limits of what is possible!

In this demo, our vehicle can be seen performing autonomous driving at very high speeds, causing it to both skid and drift at turns, while also avoiding obstacles. At such speeds, given the inherent dynamics of the vehicle platform used, it is very easy for the vehicle to topple.

The reinforcement learning based motion planning and decision making framework that is being demoed here is tasked with ensuring obstacles avoidance without compromising on the speed, to an extent possible, and to drive the vehicle as fast as possible. This is evident towards the end of the video, where it can be seen that our vehicle avoided static obstacles while drifting. This demonstrates the level of sophistication and agility in our framework to ensure proper control of the autonomous vehicles at high-speeds.

The use cases are many; to begin with, our generic off-roads autonomous driving research focuses on enabling autonomous navigation in previously unknown and unseen environments, while ensuring mathematical completeness guarantees. Such agility can also help on-road autonomous vehicles to deal with unforeseeable corner cases or sudden appearance of obstacles in its tracks, at high-speeds.

The underlying research is still far from over, and over the next 3-4 months, we will be demonstrating abstract representation being learned by our multi-RL agents based framework (under progress) to ensure computation of the cost of the terrain without any labelled data, where multiple agents learn to control / regulate different aspects of autonomous navigation, to ensure safe and robust navigation, both on- and off-roads.

All the people on the ground, who participated in the demo, were trained safety professionals.

This demo marks the beginning our generic off-roads autonomousdriving research, at very high speeds, i.e., near-drift speeds, without requiring HD maps, with driving policies being learned via unsupervised and reinforcement learning.

During navigation our vehicle consistently maintained speeds in the range of 50-63 KM/H. For human experts, typical driving speeds range around 45 KM/H, as beyond that there is a high probability of vehicle skidding and drifting (i) around turns, and (ii) when brakes are applied.

The terrain was rutted, and highly uneven, with large and sharp pieces of rocks scattered all around. Any error would have resulted in a near-fatal incident for the safety driver (myself in this case). At high-speeds, there are high chances of vehicle hitting those pieces of rocks around the road, and getting rattled and thrown off-balance. It can be seem from the intermittent outside camera clips that the vehicle is drifting at many points, but still able to correct its course and controls to gain balance. This demo was done in the city of Bhopal in India.

This demo in particular made use of classical reinforcementlearning for learning control and motion planning policies. The underlying research is a long-shot cutting-edge fundamental RD program at Swaayatt Robots, with a goal of allowing autonomousvehicles to navigate previously unknown and unseen terrains, with no HD maps, for military and general off-roads autonomous navigation applications.

One critical component of this research, that makes it different from any off-roads autonomy program out there in the world is that learning is purely unsupervised, with no labelled data required.

Going forward, we will showcase driving policies, how they visualize the environment, the learned abstract representation of the world for end-to-end holistic decision making and motion planning, and much more.

Autonomous Driving at scale, through complex geometric terrain and road network, negotiating stochastic complex adversarial traffic dynamics, on the roads in India.

This demo showcases our autonomous vehicle starting from a densely cluttered region, planning its route to the Shree Ram Mandir (its final destination) in the Barkheda region, in the city of Bhopal.

Our autonomous driving technology enables smooth navigation of our autonomous vehicle through the static and dynamic traffic encountered on the road.

This demo utilized our motion planning and decision making algorithmic framework that allows autonomous vehicles to negotiate bidirectional traffic on single lane roads, that we have been demonstrating since October 2023. The vehicle abides by the traffic rules and remains on the left side of the road when the road is wide enough, and engages open environment navigation when faced with a tight single-lane road.

The route planning computation happens over our custom GPS maps of the environment. This demo highlights our capabilities to perform sustainable autonomous driving at scale.

Next, we will be demonstrating end-to-end learned policies via inverse reinforcement learning for traffic intersection and roundabout navigation of autonomous agents in the presence of complex traffic dynamics with stochastic adversarial agents.

As we further our Level 5 autonomous driving research and scale up the technology, including sustainable last mile autonomy operational capabilities, we begin our day-time operations on regular urban roads in India. This is to ensure that our autonomous driving technology can work globally and can negotiate any kind of environments and traffic dynamics.

We will gradually increase the complexity of the traffic dynamics and the environments that our autonomous vehicles are tasked to negotiate.

In this demo, the vehicle was tasked with navigating a regular 4-lane city road, with the end goal being able to park itself in a very constrained unstructured parking spot. Often, even on 4-lane roads, vehicles, bicyclists, and autos can come from the wrong side. We mimic this stochastic traffic pattern by having motorcyclists come from the wrong side and not yield to our vehicle -- leaving it to our vehicle to decide whether to come to a standstill or to yield from its current driving lane and perform an avoidance maneuver.

At the end, we demonstrate last mile autonomy capability by having our autonomous vehicle park itself in a very unstructured parking spot, with bikes parked randomly in the area, outside of our office. We had earlier demonstrated such capabilities in structured campus environments in our demos in January and February 2023. As can be seen from the demo, the vehicle executed these tasks in a generic open environment, while also negotiating a traffic intersection and crossing the divider on the 4-lane road.

This is only the beginning of our day-time operations on roads in India.

In this demo, we showcase our autonomous vehicle avoiding piles of soil and rocks dumped on the road, for the purpose of construction.

Autonomous driving off-roads is an extremely challenging problem. Our autonomous vehicle navigate through a construction zone off-roads. Navigating such terrains autonomously is unthinkable and is beyond the capability of any startup anywhere in the world.

Our autonomous vehicle had never seen this kind of an environment before, and the obstacles detection algorithm was also never trained with any such scenarios or for any such tasks.

Negotiation of a construction site, including with very well structured traffic-cones, is usually considered as a challenge by the autonomous driving industry in the West. For on-roads autonomous driving, perhaps high-definition or sparse maps of the environments could be used, as it typically the case with the companies in the West.

Over here, for the construction work, the piles of cones of rocks and soil were dumped on the road. Such a scenario is extremely difficult to navigate as an autonomous vehicle must determine how much to deflect because of the impeding pile of soil and rocks when no specific boundaries information is available on either side.

For off-roads autonomousdriving, any approach that relies on high-definition maps is not going to scale up. Off-roads autonomous vehicles must be equipped with the level of on-board intelligence that they can negotiate any scenario, any challenge, and any obstacles coming from the other direction at regular speeds.

We have earlier demonstrated capability to negotiate inflow of traffic off-roads, that is unique to Swaayatt Robots. Acquiring the ability to negotiate construction sites like this is a new frontier altogether.

While earlier, in 2017, we demonstrated our perception algorithmic framework, Deep Energy Maps, to detect the boundaries of drivable regions both on- and off-roads, even scaling such algorithms for these kinds of tasks would be a daunting undertaking. Our approach towards off-roads autonomous driving is going to kill the requirement of specific perception algorithmic pipelines altogether.

This motion planning and decision making framework is being scaled up further, and going forward we will showcase its strength, trained via our highly data-efficient forward and inverse reinforcement learning  and unsupervised learning approaches -- developing embodied intelligence for such navigation tasks.

Autonomous driving through dense dynamic traffic, with extremely tight-complex-stochastic traffic dynamics on sub-urban roads, connecting to open ground, with absolutely zero traffic rules. This is the most heavily cluttered environment where we have tested our autonomous driving technology thus far, presenting many of the adversarial negotiation scenarios as well, throughout the navigation. This demo was done at the Mata Baglamukhi Mandir campus in the city of Nalkheda, in MP, and was done in the presence of heavy police forces that were deployed that day on the ground, as can be seen in our demo.

Our autonomous vehicle starts from the temple with a generic open environment, with zero traffic rules, with very narrow corridors created out of barricades for vehicle movement by the security forces. In this corridor no two vehicles can pass through at the same time, and the vehicle was tasked with driving through this corridor, while negotiating its way from any traffic, two-wheeler, or pedestrians it faces, with the dense presence of bikes and cars on either side, presenting a very challenging environment for autonomousvehicles.

The vehicle exits the open area, and then assumes generic dual lane navigation, avoiding both static and dynamic obstacles, before encountering a police check-post, where the vehicle is supposed to wait if the barricade is closed and proceed if open. Upon exiting the police check-post, the vehicle negotiates a traffic intersection with stochastic and adversarial driving behavior of other vehicles on the road.

Autonomous driving through extremely tight and dynamic environments with complex, stochastic, and adversarial traffic dynamics, or simply through absolute chaos, on sub-urban unstructured roads in India. These demos tested our motion planning and decision-making framework to its limits, showcasing the robustness of our autonomous driving technology in negotiating such traffic dynamics with ease. This demo was done on a very narrow road, suited mostly for one-way navigation, but as is customary in India, bidirectional traffic is active on such narrow roads.

It can be seen throughout the navigation that the incoming vehicles didn't allow any gaps for our autonomous vehicles, forcing it to negotiate passively-aggressively its path through the chaos. Furthermore, obstacles overtaking us didn't follow any rules either, and zig-zagged and moved in a crisscross fashion, challenging our motion and behavior planning software, which negotiated all such scenarios with ease.

There were only two points where our vehicle came to a halt when two girls on a two-wheeler didn't stop and just kept on navigating, despite our vehicle being closer to the narrow passage and having the right of way, and despite a bike being parked over there by someone, making it a very challenging scenario both for the humans and for the decision making autonomous agent(s).
 

High-speed off-roads autonomous driving, with bidirectional dynamic traffic, paving the road for military and several other off-roads autonomous operations.

The demo and tech capabilities shown here address several of the requirements of not just the Indian armed forces but also the solutions to various problems that are currently a topic of active R&D for the Defense Advanced Research Projects Agency (DARPA) and the United States Department of Defense .

In this demo, it can be seen that our autonomous vehicle can seamlessly navigate an off-roads environment, along with negotiating bidirectional traffic that didn't necessarily follow the traffic rules. Whenever our vehicle senses that an incoming vehicle is following the rule and is navigating on the correct side, it avoids them from the left, and when it senses the other vehicle is coming from the wrong side, rather than getting confused and coming to a halt, it avoids it from the opposite side. This is soft-traffic-rules constraints adherence.

Currently, our vehicle can perform this at 44 KM/H, where it slows down for negotiation, coming to safe speeds first. This framework is being scaled up further for 60-80 KM/H relative-speed avoidance maneuvers on such roads.
 

Autonomous driving through tight, dynamic, stochastic, and adversarial traffic dynamics on sub-urban roads in India and partially unstructured environments.

This demo showcases the robustness of our motion planning and decision-making algorithmic frameworks in enabling autonomous driving seamlessly through such traffic and environmental scenarios. The vehicle starts from a generic open environment at the temple, with no traffic rules to abide by. It then exits the region and assumes a generic autonomous navigation behavior, negotiating complex traffic scenes. At various points it can be seen that the other vehicles (bikes, autos, bicyclists, and cars) didn't abide by any traffic rules, or respect right-of-way, and moved in a crisscross fashion, presenting adversarial scenarios, challenging our autonomous vehicle to take care of the collision avoidance.

This classical motion planning and decision-making algorithmic framework is being further scaled up with deep reinforcement learning, which will practically solve the sub-urban traffic dynamics and environment negotiation for autonomous vehicles in India and worldwide.

Off-roads autonomous driving at high-speeds is an extremely challenging problem. On 21st of January we did such a demo on our new autonomous driving test vehicle, an Autonomous Thar.

Earlier last year we showcased off-roads autonomous driving through dense fog (January 2023), high-speeds off-roads autonomous driving via our new reinforcement learning based planning algorithm (April 2023), bidirectional traffic negotiation off-roads with narrow avoidance (September 2023).

Our off-roads autonomous driving research is targeted towards enabling autonomous vehicles to traverse such terrains with Embodied Intelligence, doing both holistic and non-holistic inverse RL. The objective is to kill the perception algorithmic pipeline altogether for on-roads autonomous driving, to achieve sustainable, cost-effective, and operationally efficient Level-5 autonomous driving by 2026/27.

Expanding on our objective of bringing Level-5 autonomous driving in the world's most adversarial-complex-stochastic of traffic-dynamics and environmental conditions, we present autonomous driving in very dense fog at night, with negotiation of incoming traffic on a single lane road, in India.

This bidirectional negotiation, i.e., single lane road traffic negotiation, requires the motion planning and decision-making algorithms of autonomous vehicles to be able to understand when to shift off-road, and how much to shift, depending on how much the incoming vehicle is ready to yield, and then make appropriate behavioral decisions. We showcased this capability for the first time in the field of autonomous driving in our October 2023 demo.

In our earlier demos, we have shown our autonomous vehicles negotiating very complex, stochastic, and adversarial traffic-dynamics, like our Level-5 demo at the Toll-Plaza last month, and then the Bhopal city demo on the night of the new year.

In this demo, we present autonomous driving at night through very dense fog while negotiating incoming traffic on a single-lane road. In such scenarios, the incoming traffic vehicle's headlights completely blind the ego-vehicle's visual sensors, like the cameras.

While robust detection of obstacles at night using only the cameras, including perceiving their shape and estimating their depth, is an active area of unsupervised deep learning research at Swaayatt Robots (स्वायत्त रोबोट्स), to showcase our capabilities to work with fused information of visual sensors, we made use of a LiDAR in this demo.

During the initial 15 seconds, it can be seen to avoid the incoming vehicle, which didn't yield any room for our vehicle; our vehicle decided to shift off-roads, to enable a successful negotiation without coming to a complete halt.

Again, this is the first of its kind of a demo, where an autonomous vehicle is negotiating a single-lane road with bidirectional traffic at night, through FOG. It is second only to our daytime demo of complex negotiation in October of last year.

In this demo our autonomous vehicle can be seen negotiating very complex and adversarial traffic situations, where the surrounding vehicles and bikes randomly cross its path and switch lanes in a stochastic manner without adherence to any traffic rules. Our vehicle successfully negotiated such situations with ease while adhering to its lane.

Presenting autonomous driving in highly complex, stochastic and adversarial traffic-dynamics, on the roads in India. Over the course of last year, we enabled autonomous driving in conditions and in situations no one in the autonomous driving industry considered was possible, and in a country like India, where the contemporary belief was that autonomous vehicles are an impossibility.

In this demo we showcase autonomous driving for toll-gate crossing and complex traffic negotiation at such intersections, while dealing with the highly stochastic behaviours of large trucks.

This is the world's first and the only demo of autonomous vehicles learning to perform maneuvers of such a complexity.Our vehicle can be seen entering the toll-gate region through a highway, and then negotiating bidirectional traffic dynamics in an open area with literally no driving rules to adhere to in such regions. Truck driver often park their trucks randomly at night in such areas, which can be seen in the video, with numerous trucks parked there. Furthermore, our vehicle had to decide which toll-gate-passage to commit to, while negotiating the criss-cross navigation pattern of the trucks that over-took us.

Bringing sustainable autonomous driving to India and to the world. This marks the beginning of our trials of autonomous vehicles to bring autonomous transportation to the part of the world no one expected, i.e.,in India.

Currently our city trials are restricted to performing autonomous driving operations at night,an ongoing endeavour to bring safe,scalable,and cost-effective autonomous mobility globally. This demo shows autonomous navigation at night on the city roads in India, in the city of Bhopal.Earlier in the month of October we introduced bidirectional negotiation on single lane roads to the world of autonomous driving -- a capability no other autonomous mobility company has demonstrated. While the technology is almost-ready for city traffic as well, we are staring our operations first with moderate traffic conditions at night.

While the city roads at night are sparsely populated, they have their unique challenges. For example, one can notice an auto coming in from the wrong direction. Similarly, bikes at night can cross the roads in very stochastic manner. Our ability to negotiate bidirectional traffic on single lane roads makes it look like a cake-walk. Whereas, typically,autonomous vehicles that are too much dependent on maps can get stuck very easily in such scenarios --like Cruise and Waymo (who are busy in getting their licenses suspended, with their approach to autonomous driving).

We will keep on bridging the gap between our chaotic traffic autonomy demos during the day on highways and our demos in the city at night, with ability to negotiate very dense night traffic in the coming weeks.Furthermore, with our unique unsupervised learning and reinforcement learning based approach to further enhance the bidirectional negotiation and overtaking policies at Swaayatt Robots ,we will showcase a one-of-its-kind demo in the month of January 2024.

In this demo, we showcase autonomous driving at high speeds on a single-lane road with bidirectional traffic negotiation. When a vehicle approaches from the opposite end of the road, our autonomous vehicle is required to move to the side of the road.

This is the world's first demo showcasing such negotiation capabilities in the context of autonomous vehicles. We have developed a novel classical motion planning and decision-making algorithmic framework. We had previously demonstrated this framework off-road in September at low speeds. The framework uses multiple horizons for planning and decision-making, interleaving the decisions of multiple agents to allow our autonomous vehicle, Swaayatt Robots (स्वायत्त रोबोट्स), to execute such behavior.

We are working towards scaling this framework through end-to-end deep inverse reinforcement learning, which will be showcased in January 2024.

The autonomous driving industry, predominantly in North America and Europe, focuses on perfect maps-based avoidance with a significant margin of error on broad roads. Faced with scenarios like the ones shown here, they often stop indefinitely or yield. In contrast, our research effort aims to enable autonomous vehicles to make decisions, such as when to yield, when to go off-road, or whether to completely stop and wait for the road to clear. The present demo showcases the beginning of this journey towards Level-5 autonomous driving, including negotiation at high speeds through chaotic, dense, and tight roads.

We executed this demo on the 22nd of October, 2023, on the day of Maha-Ashtami. This marks the beginning of our research on achieving speeds of 101 KM/H, which we will demonstrate in around six months. This represents the first step in the research direction, offering a glimpse of the future of autonomous driving.

This demo showcases autonomous driving in the wild, where our autonomous vehicle was tasked with avoiding generic static and dynamic obstacles in its path -- with the margin of error being very low. At some points, our autonomous vehicle had few tens of centimetre of margin, from the impending obstacles, with very-rough and bumpy terrain ahead of it. For a significant portion of the navigation, the vehicle had to negotiate bi-directional traffic on a single-lane off-road -- although we developed novel motion planning and decision making frameworks to enable this, around January we will showcase this capability at very high-speeds, via a multi-agent intent analysis and bi-directional negotiation policy, learned via deep unsupervised learning + reinforcement learning. This demo made use of very sparse stochastic HD maps, for stochastic projection of boundary information. In the coming weeks and months we will showcase autonomous driving without any HD map off-roads, showcasing the agent building the cost-map of the navigable region via unsupervised deep learning, including building an inherent understanding of the boundaries of the environment, utilizing only GPS-nodes as waypoints. Our research in off-roads autonomous driving in general focuses on achieving the end-goal of enabling autonomous vehicles traverse any terrain without using HD maps, learning a driving policy via unsupervised machine learning and reinforcement learning -- pushing the boundary of what is considered possible in autonomous driving. This will eventually trickle down to on-road autonomous driving, where we will showcase elimination of the key perception algorithms altogether -- road detection, lane detection, terrain-map, and perhaps obstacles detection as well. In the pursuit of achieving Level-5 autonomy we will be doing a 100 KM/H autonomous driving demo, showcasing: (i) Generic bi-directional negotiation at high-speeds on a single lane road, (ii) over-taking and ability to abort like human drivers do, and (iii) high-speed navigation without any maps. As a first step in this endeavor we will be executing a similar demo on a single lane on-road environment at high-speeds, where often to avoid the on-coming obstacle, the vehicle will have to drift off-roads -- utilizing the stack we have built for generic off-roads autonomous driving, which will be soon scaled with unsupervised learning.

In this demo, our vehicle can be seen traversing through dynamic, unpredictable, and tight obstacles while also ensuring soft constraints of its lane and the environment.

Our research at Swaayatt Robots (स्वायत्त रोबोट्स) in several different fronts of theoretical computer science and applied mathematics, including, but not limited to #reinforcementlearning, transfer learning, #deeplearning, #machinelearning, motion planning, SLAM and mapping, computer vision, stochastic processes, manifold alignment, that we have been doing over the years, and demonstrated through 60+ demos over the past few years, is now taking a definitive shape to enable sustainable autonomous driving at scale.

Next will showcase our R&D in enabling mapping at a very large scale via both contextual and symbolic transfer of knowledge across the maps, and also showcase generalized obstacles avoidance off-roads, first via using perception algorithms, and then by eliminating several of the perception algorithms, without any HD maps, via the novel unsupervised learning based decision making framework we have been working on.

We expect to open autonomous driving, and #autonomousvehicles experience to the public in India in the month of October. This will be historic in the context of autonomous driving. Very soon we will execute high-speed autonomous driving on a mountainous terrain as well.
 

Navigating at night at high-speeds (max speed during the demo 64 KM/H), safely executing lane keeping, merging into a traffic intersection, and safely executing round-about navigation policy.

While the roads over here were a bit well-paved, given our autonomous lane detection and generation algorithm, which is being scaled, for both day and night operations, that automatically generates lane markers on-the-fly in the absence of lanes markers in the visual-sensory data, we are able to tackle both structured- and unstructured-roads (as we had demoed in the past). Now this is being scaled for state-wide and country-wide operations.

Earlier we had demoed off-roads autonomous driving, via #reinforcementlearning, and when that research is merged with our on-roads autonomous driving in the coming weeks, we will have scalable #autonomousvehicles capable of negotiating any kind of roads and environmental situations.

Next we will be showcasing autonomous driving at night through moderate traffic conditions of the Indian cities, and very-soon during the day-time as well.

Autonomous driving off-roads at high-speeds, a one of its kind of a demo, handling such levels of uncertainty and unstructured off-road.

The core motion planning and decision making in this demo is being governed by a reinforcement learning agent. The vehicle reached a top speed of 44 KM/H, which is near the limit of humans driving safely on this road, without risking drift or side-skids at turns.

At Swaayatt Robots (स्वायत्त रोबोट्स) we have initiated a research project, with the long term goal being, at least for off-roads, to enable the autonomous vehicle make decisions about motion and behavior without relying on explicit computation of the various perceptual features, for example, localization against HFMs or detection of delimiters or obstacles.

This demo showcases a glimpse of the classical variant of the underlying research put into action. In the end, in a few months hereafter, this novel decision making and motion planning algorithmic framework will change the off-roads #autonomousdriving paradigm and how #autonomousvehicles compute actions or make navigation decisions.

The deep #reinforcementlearning variant of this algorithmic framework will completely eradicate the need for explicit perception algorithms, and autonomous vehicles will be able to make a gist of their environment and take behavioral actions, without explicitly detecting road boundaries or delimiters as well as obstacles and computing their intentions. This research will then be utilized on-roads to achieve scalable Level-5 autonomous driving.

This demo used explicit obstacles detection, which worked robustly on-roads, as well as two LiDARs and cameras.

It can be seen that our autonomous driving software, which uses reinforcement learning, enabled our autonomous vehicle to navigate through very-tight regions in the campus. Executing such tight turns required fundamentally innovating the controller algorithm on the fly, within a few hours -- I had to use my several years of mobile robotics and motion planning experience to get this done rightly. Our vehicle had driven all the way from Bhopal to Mussoorie, through rain, dust, and fog, around 1100 KM one way trip. As a result, on the 20th, our braking system and many of the cameras failed to function. Thus, we were not able to showcase our full capabilities on the 20th. However, after much effort, we were able to activate our braking system on the 21st of March, and again did the demo, that is being showcased here. This was an unknown campus to us, and with a few hours of fine-tuning, we were able to transfer our campus autonomous driving software to function at the previously unseen LBSNAA campus. This again highlights the readiness of our campus autonomous driving software to execute autonomous driving anywhere with minimal efforts.

Autonomous driving through generalized traffic with autonomous parking maneuver via reinforcement learning-based driving policy. We are going to begin executing autonomous driving on urban roads, i.e., city roads with unruly traffic, in India starting March/April, and this demo is a part of a series of demos and trials we will be doing along the way, to both achieve autonomous urban driving and 100 KM/H L-4 autonomous driving in India. In this demo, our autonomous vehicle is tasked with negotiating and avoiding generalized uncontrolled traffic with an end goal of parking at a designated spot with near-end-effector constraints. The complexity of this task is humongous, given the unpredictable pattern of the traffic and pedestrians in an open environment -- there are no traffic rules that can provide insights into the behavior of pedestrians or other vehicles. This was achieved through an autonomous driving policy learned via reinforcement learning. Earlier in 2017 as well, we had demoed a multi-agent reinforcement learning-based framework to perform autonomous navigation through stochastic tight roads in India with generalized traffic. The present demo and the earlier 2017 demo make us the only autonomous vehicles startup to put a full-fledged autonomous SUV on Indian roads.

Autonomous driving off-roads with very thick fog! This capability makes us the 3rd startup/company in the world to have both on- and off-roads autonomous driving capabilities, after Wayve AI (2018) and Oshkosh Defense. Furthermore, we are the only startup in the world to demo autonomous driving capabilities in India and actively pursuing Level-5! The ongoing DARPA RACER program's objective is to drive off-roads at human-driven speeds. We acknowledge the work, specifically by University of Washington's team, where they are building an explicit cost-map to drive off-roads. Our work is more unsupervised. We are perfecting the off-road autonomous driving policy, including optimal speed control over off-roads, without any prior map of the environment, and without any explicit algorithm for processing of the visual sensory information, and we should be able to demo this capability in the coming weeks in the form of a PoC. This demo utilized one VLP-32C LiDAR from Velodyne. Out of the 25+ trials we did, there was one point where manual intervention was felt necessary, as this was a single-lane road with bi-directional traffic. This kind of negotiation is an active area of research at Swaayatt Robots (स्वायत्त रोबोट्स), and we are learning a policy to solve this problem.

At Swaayatt Robots, we have initiated autonomous driving trials on city roads in India during the nighttime, and our plan is to expand these trials to daytime operations by Q2-2023, even amidst the chaotic traffic. During these trials, we will be showcasing the following: 1. The decision-making capabilities of our reinforcement learning-based autonomous driving policies. 2. Navigation capabilities with stochastic localization against high-fidelity maps. 3. High-precision localization against high-fidelity maps for last-mile autonomy. 4. Perception techniques using both LiDAR and non-LiDAR based approaches. These demonstrations represent significant steps in advancing our autonomous driving technologies. Furthermore, our autonomous driving trials on city roads in India serve as a testament to the adaptability of our technology in diverse and complex real-world environments. Navigating through the bustling streets, our autonomous vehicles showcase their ability to respond to unpredictable scenarios, ensuring safety and efficiency even in challenging traffic conditions. As we transition to daytime trials, our commitment to pushing the boundaries of autonomous driving remains unwavering. We envision a future where our technology not only enhances convenience and safety on the roads but also contributes to the evolution of urban mobility, making transportation more sustainable and accessible for all. With continuous innovation and rigorous testing, we are excited to bring the promise of autonomous driving closer to reality.

Autonomous driving navigating through tight obstacles validating two of the motion planning algorithms that we are furthering at Swaayatt Robots, and for which will be submitting three papers, two of those perhaps in Robotics Science and Systems conference, and filing provisional patent for the one. This demo required integration of active and passive perception algorithmic pipelines, and by the end of 2022, we will finish most of the campus autonomous driving software, with parking feature, which will go on and become the first commercial offering from Swaayatt. In this demo, one of the motion planning algorithms, that is being furthered with reinforcement learning, is the one I originally invented when I was an undergrad at IIT Roorkee. It is the first time we are testing it on autonomous vehicles as it guarantees safety mathematically for mobile robots exhibiting kino-dynamics, such as a campus vehicle navigating at manageable speeds. The second is an algorithm that I developed with Zvi Shiller during my internship under him, and have significantly improved it in 2015. One of the versions of this algorithm was published in the International Journal of Robotics Research in 2013, and hopefully, being furthered by reinforcement learning, we will be able to submit a revised version of the same. This demo is also a step towards executing a major demo around Diwali, and ensuring that even at high speeds, the vehicle can properly slow down and avoid cluttered environments without diverging too much from its lane, and not stop too much unnecessarily, as most of the autonomous vehicles do in the US and in Europe -- which is not a proper use of the capabilities of autonomous vehicles.

a challenge we have taken on to develop an affordable system, consequently limiting the vehicle's speed. For the product launch, we ma   For the second time now, we have commenced comprehensive testing of our reinforcement learning-based ADAS (Advanced Driver Assistance System) and autonomous driving software on uncontrolled roads in India. Back in 2017/18, we had previously demonstrated the effectiveness of the RL-first approach to autonomous driving on the unpredictable roads of India. Currently in the prototyping stage, the software being showcased in this demo will enable ADAS, lane-keeping, and other L2 autonomous driving features on roads as unstructured as those found in India, all without the need for HD Maps. In this demonstration, the vehicle's core lane-keeping operation relies solely on a single forward-facing GigE-IP camera, out of the six cameras mounted on the roof-top. Our lane detection and generation software, which has been demonstrated multiple times since 2017, enables autonomous vehicles to both detect and generate lane boundaries on-the-fly, actively perceiving the drivable region. A reinforcement learning-based planning and behavior generation software then assumes control of the vehicle. Depth perception is achieved using only a single camera,y opt to utilize a stereo camera system.

This demo showcases our autonomous driving software in the prototyping stage. When productized at Swaayatt Robots, it will enable (i) campus autonomous driving, (ii) autonomous forward and reverse parking with near end-effector constraints satisfaction, (iii) execution of tight S-Curves, and (iv) U-Turns, with intelligence to decide which spot to park at, based on the clutter and the contextual understanding of its environment. In this demo the environment is divided into 3-zones, each zone having some drive lanes and some park-lanes. The core motion planning software plans an optimized trajectory to meet the end-effector constraints and has to decide on the fly which divider passageway is to be selected to transition to the next zone. The planning is online, as opposed to off-line/global planning approach adopted by the industry. Next we would be blending this with highly optimized #reinforcementlearning (RL) based decision-making framework, using Semi Markov Decision Processes (S-MDPs), that would automatically be able to decide whether to stop or not for other vehicles, and when to initiate reverse direction navigation depending on the contextual understanding of the environment. One of my core focuses since undergrad is autonomous navigation in unknown and unseen environments via reinforcement learning. Thus, the learned RL-agents' policies will be independent of the environment / simulation data it is trained in, and would be able to work in any campus environment, with stochastic localization against high-fidelity maps (like GPS localization) in such environments. Further, to scale-up last mile autonomy, in this demo we also show the wire-frame high-fidelity map of the environment, which is useful for such last mile #autonomousmobility tasks.

Autonomous driving, executing U-Turns at high-speeds, i.e., near-drift speeds, and lane-change operations, via reinforcement learning. This video demonstrates Swaayatt Robots (स्वायत्त रोबोट्स) autonomous driving technology enabling (i) high-speed maneuvers, (ii) lane-keeping, and (iii) lane-shift operations. The core navigation policy for the RL agent, which is executed by our autonomous vehicle, was majorly learned in simulations and transferred to the real-world. The vehicle entered the U-Turn at 45 KM/H (which is near-drift speed given the tight-turn and the vehicle's dynamics) and executed it precisely, and thereafter performed a lane switching maneuver to reach its target parking stop. This demo was also performed to machine learn and iteratively-calibrate our system, for the 100 KM/H autonomous driving (which we will be executing within the next 7 months), where the vehicle will have to execute many such high-speed turns on mountainous terrain.

Swaayatt Robots' Autonomous vehicle was featured on Zee Hindustan news channel . In this video, CEO Mr. Sanjeev Sharma is demonstrating the autonomous driving of their vehicle.