Autonomous driving through dense dynamic traffic, with extremely tight-complex-stochastic traffic dynamics on sub-urban roads, connecting to open ground, with absolutely zero traffic rules. This is the most heavily cluttered environment where we have tested our autonomous driving technology thus far, presenting many of the adversarial negotiation scenarios as well, throughout the navigation. This demo was done at the Mata Baglamukhi Mandir campus in the city of Nalkheda, in MP, and was done in the presence of heavy police forces that were deployed that day on the ground, as can be seen in our demo.

Our autonomous vehicle starts from the temple with a generic open environment, with zero traffic rules, with very narrow corridors created out of barricades for vehicle movement by the security forces. In this corridor no two vehicles can pass through at the same time, and the vehicle was tasked with driving through this corridor, while negotiating its way from any traffic, two-wheeler, or pedestrians it faces, with the dense presence of bikes and cars on either side, presenting a very challenging environment for autonomousvehicles.

The vehicle exits the open area, and then assumes generic dual lane navigation, avoiding both static and dynamic obstacles, before encountering a police check-post, where the vehicle is supposed to wait if the barricade is closed and proceed if open. Upon exiting the police check-post, the vehicle negotiates a traffic intersection with stochastic and adversarial driving behavior of other vehicles on the road.

Autonomous driving through extremely tight and dynamic environments with complex, stochastic, and adversarial traffic dynamics, or simply through absolute chaos, on sub-urban unstructured roads in India. These demos tested our motion planning and decision-making framework to its limits, showcasing the robustness of our autonomous driving technology in negotiating such traffic dynamics with ease. This demo was done on a very narrow road, suited mostly for one-way navigation, but as is customary in India, bidirectional traffic is active on such narrow roads.

It can be seen throughout the navigation that the incoming vehicles didn't allow any gaps for our autonomous vehicles, forcing it to negotiate passively-aggressively its path through the chaos. Furthermore, obstacles overtaking us didn't follow any rules either, and zig-zagged and moved in a crisscross fashion, challenging our motion and behavior planning software, which negotiated all such scenarios with ease.

There were only two points where our vehicle came to a halt when two girls on a two-wheeler didn't stop and just kept on navigating, despite our vehicle being closer to the narrow passage and having the right of way, and despite a bike being parked over there by someone, making it a very challenging scenario both for the humans and for the decision making autonomous agent(s).
 

High-speed off-roads autonomous driving, with bidirectional dynamic traffic, paving the road for military and several other off-roads autonomous operations.

The demo and tech capabilities shown here address several of the requirements of not just the Indian armed forces but also the solutions to various problems that are currently a topic of active R&D for the Defense Advanced Research Projects Agency (DARPA) and the United States Department of Defense .

In this demo, it can be seen that our autonomous vehicle can seamlessly navigate an off-roads environment, along with negotiating bidirectional traffic that didn't necessarily follow the traffic rules. Whenever our vehicle senses that an incoming vehicle is following the rule and is navigating on the correct side, it avoids them from the left, and when it senses the other vehicle is coming from the wrong side, rather than getting confused and coming to a halt, it avoids it from the opposite side. This is soft-traffic-rules constraints adherence.

Currently, our vehicle can perform this at 44 KM/H, where it slows down for negotiation, coming to safe speeds first. This framework is being scaled up further for 60-80 KM/H relative-speed avoidance maneuvers on such roads.
 

Autonomous driving through tight, dynamic, stochastic, and adversarial traffic dynamics on sub-urban roads in India and partially unstructured environments.

This demo showcases the robustness of our motion planning and decision-making algorithmic frameworks in enabling autonomous driving seamlessly through such traffic and environmental scenarios. The vehicle starts from a generic open environment at the temple, with no traffic rules to abide by. It then exits the region and assumes a generic autonomous navigation behavior, negotiating complex traffic scenes. At various points it can be seen that the other vehicles (bikes, autos, bicyclists, and cars) didn't abide by any traffic rules, or respect right-of-way, and moved in a crisscross fashion, presenting adversarial scenarios, challenging our autonomous vehicle to take care of the collision avoidance.

This classical motion planning and decision-making algorithmic framework is being further scaled up with deep reinforcement learning, which will practically solve the sub-urban traffic dynamics and environment negotiation for autonomous vehicles in India and worldwide.

Off-roads autonomous driving at high-speeds is an extremely challenging problem. On 21st of January we did such a demo on our new autonomous driving test vehicle, an Autonomous Thar.

Earlier last year we showcased off-roads autonomous driving through dense fog (January 2023), high-speeds off-roads autonomous driving via our new reinforcement learning based planning algorithm (April 2023), bidirectional traffic negotiation off-roads with narrow avoidance (September 2023).

Our off-roads autonomous driving research is targeted towards enabling autonomous vehicles to traverse such terrains with Embodied Intelligence, doing both holistic and non-holistic inverse RL. The objective is to kill the perception algorithmic pipeline altogether for on-roads autonomous driving, to achieve sustainable, cost-effective, and operationally efficient Level-5 autonomous driving by 2026/27.

Expanding on our objective of bringing Level-5 autonomous driving in the world's most adversarial-complex-stochastic of traffic-dynamics and environmental conditions, we present autonomous driving in very dense fog at night, with negotiation of incoming traffic on a single lane road, in India.

This bidirectional negotiation, i.e., single lane road traffic negotiation, requires the motion planning and decision-making algorithms of autonomous vehicles to be able to understand when to shift off-road, and how much to shift, depending on how much the incoming vehicle is ready to yield, and then make appropriate behavioral decisions. We showcased this capability for the first time in the field of autonomous driving in our October 2023 demo.

In our earlier demos, we have shown our autonomous vehicles negotiating very complex, stochastic, and adversarial traffic-dynamics, like our Level-5 demo at the Toll-Plaza last month, and then the Bhopal city demo on the night of the new year.

In this demo, we present autonomous driving at night through very dense fog while negotiating incoming traffic on a single-lane road. In such scenarios, the incoming traffic vehicle's headlights completely blind the ego-vehicle's visual sensors, like the cameras.

While robust detection of obstacles at night using only the cameras, including perceiving their shape and estimating their depth, is an active area of unsupervised deep learning research at Swaayatt Robots (स्वायत्त रोबोट्स), to showcase our capabilities to work with fused information of visual sensors, we made use of a LiDAR in this demo.

During the initial 15 seconds, it can be seen to avoid the incoming vehicle, which didn't yield any room for our vehicle; our vehicle decided to shift off-roads, to enable a successful negotiation without coming to a complete halt.

Again, this is the first of its kind of a demo, where an autonomous vehicle is negotiating a single-lane road with bidirectional traffic at night, through FOG. It is second only to our daytime demo of complex negotiation in October of last year.

In this demo our autonomous vehicle can be seen negotiating very complex and adversarial traffic situations, where the surrounding vehicles and bikes randomly cross its path and switch lanes in a stochastic manner without adherence to any traffic rules. Our vehicle successfully negotiated such situations with ease while adhering to its lane.

Presenting autonomous driving in highly complex, stochastic and adversarial traffic-dynamics, on the roads in India. Over the course of last year, we enabled autonomous driving in conditions and in situations no one in the autonomous driving industry considered was possible, and in a country like India, where the contemporary belief was that autonomous vehicles are an impossibility.

In this demo we showcase autonomous driving for toll-gate crossing and complex traffic negotiation at such intersections, while dealing with the highly stochastic behaviours of large trucks.

This is the world's first and the only demo of autonomous vehicles learning to perform maneuvers of such a complexity.Our vehicle can be seen entering the toll-gate region through a highway, and then negotiating bidirectional traffic dynamics in an open area with literally no driving rules to adhere to in such regions. Truck driver often park their trucks randomly at night in such areas, which can be seen in the video, with numerous trucks parked there. Furthermore, our vehicle had to decide which toll-gate-passage to commit to, while negotiating the criss-cross navigation pattern of the trucks that over-took us.

Bringing sustainable autonomous driving to India and to the world. This marks the beginning of our trials of autonomous vehicles to bring autonomous transportation to the part of the world no one expected, i.e.,in India.

Currently our city trials are restricted to performing autonomous driving operations at night,an ongoing endeavour to bring safe,scalable,and cost-effective autonomous mobility globally. This demo shows autonomous navigation at night on the city roads in India, in the city of Bhopal.Earlier in the month of October we introduced bidirectional negotiation on single lane roads to the world of autonomous driving -- a capability no other autonomous mobility company has demonstrated. While the technology is almost-ready for city traffic as well, we are staring our operations first with moderate traffic conditions at night.

While the city roads at night are sparsely populated, they have their unique challenges. For example, one can notice an auto coming in from the wrong direction. Similarly, bikes at night can cross the roads in very stochastic manner. Our ability to negotiate bidirectional traffic on single lane roads makes it look like a cake-walk. Whereas, typically,autonomous vehicles that are too much dependent on maps can get stuck very easily in such scenarios --like Cruise and Waymo (who are busy in getting their licenses suspended, with their approach to autonomous driving).

We will keep on bridging the gap between our chaotic traffic autonomy demos during the day on highways and our demos in the city at night, with ability to negotiate very dense night traffic in the coming weeks.Furthermore, with our unique unsupervised learning and reinforcement learning based approach to further enhance the bidirectional negotiation and overtaking policies at Swaayatt Robots ,we will showcase a one-of-its-kind demo in the month of January 2024.

In this demo, we showcase autonomous driving at high speeds on a single-lane road with bidirectional traffic negotiation. When a vehicle approaches from the opposite end of the road, our autonomous vehicle is required to move to the side of the road.

This is the world's first demo showcasing such negotiation capabilities in the context of autonomous vehicles. We have developed a novel classical motion planning and decision-making algorithmic framework. We had previously demonstrated this framework off-road in September at low speeds. The framework uses multiple horizons for planning and decision-making, interleaving the decisions of multiple agents to allow our autonomous vehicle, Swaayatt Robots (स्वायत्त रोबोट्स), to execute such behavior.

We are working towards scaling this framework through end-to-end deep inverse reinforcement learning, which will be showcased in January 2024.

The autonomous driving industry, predominantly in North America and Europe, focuses on perfect maps-based avoidance with a significant margin of error on broad roads. Faced with scenarios like the ones shown here, they often stop indefinitely or yield. In contrast, our research effort aims to enable autonomous vehicles to make decisions, such as when to yield, when to go off-road, or whether to completely stop and wait for the road to clear. The present demo showcases the beginning of this journey towards Level-5 autonomous driving, including negotiation at high speeds through chaotic, dense, and tight roads.

We executed this demo on the 22nd of October, 2023, on the day of Maha-Ashtami. This marks the beginning of our research on achieving speeds of 101 KM/H, which we will demonstrate in around six months. This represents the first step in the research direction, offering a glimpse of the future of autonomous driving.

This demo showcases autonomous driving in the wild, where our autonomous vehicle was tasked with avoiding generic static and dynamic obstacles in its path -- with the margin of error being very low. At some points, our autonomous vehicle had few tens of centimetre of margin, from the impending obstacles, with very-rough and bumpy terrain ahead of it. For a significant portion of the navigation, the vehicle had to negotiate bi-directional traffic on a single-lane off-road -- although we developed novel motion planning and decision making frameworks to enable this, around January we will showcase this capability at very high-speeds, via a multi-agent intent analysis and bi-directional negotiation policy, learned via deep unsupervised learning + reinforcement learning. This demo made use of very sparse stochastic HD maps, for stochastic projection of boundary information. In the coming weeks and months we will showcase autonomous driving without any HD map off-roads, showcasing the agent building the cost-map of the navigable region via unsupervised deep learning, including building an inherent understanding of the boundaries of the environment, utilizing only GPS-nodes as waypoints. Our research in off-roads autonomous driving in general focuses on achieving the end-goal of enabling autonomous vehicles traverse any terrain without using HD maps, learning a driving policy via unsupervised machine learning and reinforcement learning -- pushing the boundary of what is considered possible in autonomous driving. This will eventually trickle down to on-road autonomous driving, where we will showcase elimination of the key perception algorithms altogether -- road detection, lane detection, terrain-map, and perhaps obstacles detection as well. In the pursuit of achieving Level-5 autonomy we will be doing a 100 KM/H autonomous driving demo, showcasing: (i) Generic bi-directional negotiation at high-speeds on a single lane road, (ii) over-taking and ability to abort like human drivers do, and (iii) high-speed navigation without any maps. As a first step in this endeavor we will be executing a similar demo on a single lane on-road environment at high-speeds, where often to avoid the on-coming obstacle, the vehicle will have to drift off-roads -- utilizing the stack we have built for generic off-roads autonomous driving, which will be soon scaled with unsupervised learning.

In this demo, our vehicle can be seen traversing through dynamic, unpredictable, and tight obstacles while also ensuring soft constraints of its lane and the environment.

Our research at Swaayatt Robots (स्वायत्त रोबोट्स) in several different fronts of theoretical computer science and applied mathematics, including, but not limited to #reinforcementlearning, transfer learning, #deeplearning, #machinelearning, motion planning, SLAM and mapping, computer vision, stochastic processes, manifold alignment, that we have been doing over the years, and demonstrated through 60+ demos over the past few years, is now taking a definitive shape to enable sustainable autonomous driving at scale.

Next will showcase our R&D in enabling mapping at a very large scale via both contextual and symbolic transfer of knowledge across the maps, and also showcase generalized obstacles avoidance off-roads, first via using perception algorithms, and then by eliminating several of the perception algorithms, without any HD maps, via the novel unsupervised learning based decision making framework we have been working on.

We expect to open autonomous driving, and #autonomousvehicles experience to the public in India in the month of October. This will be historic in the context of autonomous driving. Very soon we will execute high-speed autonomous driving on a mountainous terrain as well.
 

Navigating at night at high-speeds (max speed during the demo 64 KM/H), safely executing lane keeping, merging into a traffic intersection, and safely executing round-about navigation policy.

While the roads over here were a bit well-paved, given our autonomous lane detection and generation algorithm, which is being scaled, for both day and night operations, that automatically generates lane markers on-the-fly in the absence of lanes markers in the visual-sensory data, we are able to tackle both structured- and unstructured-roads (as we had demoed in the past). Now this is being scaled for state-wide and country-wide operations.

Earlier we had demoed off-roads autonomous driving, via #reinforcementlearning, and when that research is merged with our on-roads autonomous driving in the coming weeks, we will have scalable #autonomousvehicles capable of negotiating any kind of roads and environmental situations.

Next we will be showcasing autonomous driving at night through moderate traffic conditions of the Indian cities, and very-soon during the day-time as well.

Autonomous driving off-roads at high-speeds, a one of its kind of a demo, handling such levels of uncertainty and unstructured off-road.

The core motion planning and decision making in this demo is being governed by a reinforcement learning agent. The vehicle reached a top speed of 44 KM/H, which is near the limit of humans driving safely on this road, without risking drift or side-skids at turns.

At Swaayatt Robots (स्वायत्त रोबोट्स) we have initiated a research project, with the long term goal being, at least for off-roads, to enable the autonomous vehicle make decisions about motion and behavior without relying on explicit computation of the various perceptual features, for example, localization against HFMs or detection of delimiters or obstacles.

This demo showcases a glimpse of the classical variant of the underlying research put into action. In the end, in a few months hereafter, this novel decision making and motion planning algorithmic framework will change the off-roads #autonomousdriving paradigm and how #autonomousvehicles compute actions or make navigation decisions.

The deep #reinforcementlearning variant of this algorithmic framework will completely eradicate the need for explicit perception algorithms, and autonomous vehicles will be able to make a gist of their environment and take behavioral actions, without explicitly detecting road boundaries or delimiters as well as obstacles and computing their intentions. This research will then be utilized on-roads to achieve scalable Level-5 autonomous driving.

This demo used explicit obstacles detection, which worked robustly on-roads, as well as two LiDARs and cameras.

It can be seen that our autonomous driving software, which uses reinforcement learning, enabled our autonomous vehicle to navigate through very-tight regions in the campus. Executing such tight turns required fundamentally innovating the controller algorithm on the fly, within a few hours -- I had to use my several years of mobile robotics and motion planning experience to get this done rightly. Our vehicle had driven all the way from Bhopal to Mussoorie, through rain, dust, and fog, around 1100 KM one way trip. As a result, on the 20th, our braking system and many of the cameras failed to function. Thus, we were not able to showcase our full capabilities on the 20th. However, after much effort, we were able to activate our braking system on the 21st of March, and again did the demo, that is being showcased here. This was an unknown campus to us, and with a few hours of fine-tuning, we were able to transfer our campus autonomous driving software to function at the previously unseen LBSNAA campus. This again highlights the readiness of our campus autonomous driving software to execute autonomous driving anywhere with minimal efforts.

Autonomous driving through generalized traffic with autonomous parking maneuver via reinforcement learning-based driving policy. We are going to begin executing autonomous driving on urban roads, i.e., city roads with unruly traffic, in India starting March/April, and this demo is a part of a series of demos and trials we will be doing along the way, to both achieve autonomous urban driving and 100 KM/H L-4 autonomous driving in India. In this demo, our autonomous vehicle is tasked with negotiating and avoiding generalized uncontrolled traffic with an end goal of parking at a designated spot with near-end-effector constraints. The complexity of this task is humongous, given the unpredictable pattern of the traffic and pedestrians in an open environment -- there are no traffic rules that can provide insights into the behavior of pedestrians or other vehicles. This was achieved through an autonomous driving policy learned via reinforcement learning. Earlier in 2017 as well, we had demoed a multi-agent reinforcement learning-based framework to perform autonomous navigation through stochastic tight roads in India with generalized traffic. The present demo and the earlier 2017 demo make us the only autonomous vehicles startup to put a full-fledged autonomous SUV on Indian roads.

Autonomous driving off-roads with very thick fog! This capability makes us the 3rd startup/company in the world to have both on- and off-roads autonomous driving capabilities, after Wayve AI (2018) and Oshkosh Defense. Furthermore, we are the only startup in the world to demo autonomous driving capabilities in India and actively pursuing Level-5! The ongoing DARPA RACER program's objective is to drive off-roads at human-driven speeds. We acknowledge the work, specifically by University of Washington's team, where they are building an explicit cost-map to drive off-roads. Our work is more unsupervised. We are perfecting the off-road autonomous driving policy, including optimal speed control over off-roads, without any prior map of the environment, and without any explicit algorithm for processing of the visual sensory information, and we should be able to demo this capability in the coming weeks in the form of a PoC. This demo utilized one VLP-32C LiDAR from Velodyne. Out of the 25+ trials we did, there was one point where manual intervention was felt necessary, as this was a single-lane road with bi-directional traffic. This kind of negotiation is an active area of research at Swaayatt Robots (स्वायत्त रोबोट्स), and we are learning a policy to solve this problem.

At Swaayatt Robots, we have initiated autonomous driving trials on city roads in India during the nighttime, and our plan is to expand these trials to daytime operations by Q2-2023, even amidst the chaotic traffic. During these trials, we will be showcasing the following: 1. The decision-making capabilities of our reinforcement learning-based autonomous driving policies. 2. Navigation capabilities with stochastic localization against high-fidelity maps. 3. High-precision localization against high-fidelity maps for last-mile autonomy. 4. Perception techniques using both LiDAR and non-LiDAR based approaches. These demonstrations represent significant steps in advancing our autonomous driving technologies. Furthermore, our autonomous driving trials on city roads in India serve as a testament to the adaptability of our technology in diverse and complex real-world environments. Navigating through the bustling streets, our autonomous vehicles showcase their ability to respond to unpredictable scenarios, ensuring safety and efficiency even in challenging traffic conditions. As we transition to daytime trials, our commitment to pushing the boundaries of autonomous driving remains unwavering. We envision a future where our technology not only enhances convenience and safety on the roads but also contributes to the evolution of urban mobility, making transportation more sustainable and accessible for all. With continuous innovation and rigorous testing, we are excited to bring the promise of autonomous driving closer to reality.

Autonomous driving navigating through tight obstacles validating two of the motion planning algorithms that we are furthering at Swaayatt Robots, and for which will be submitting three papers, two of those perhaps in Robotics Science and Systems conference, and filing provisional patent for the one. This demo required integration of active and passive perception algorithmic pipelines, and by the end of 2022, we will finish most of the campus autonomous driving software, with parking feature, which will go on and become the first commercial offering from Swaayatt. In this demo, one of the motion planning algorithms, that is being furthered with reinforcement learning, is the one I originally invented when I was an undergrad at IIT Roorkee. It is the first time we are testing it on autonomous vehicles as it guarantees safety mathematically for mobile robots exhibiting kino-dynamics, such as a campus vehicle navigating at manageable speeds. The second is an algorithm that I developed with Zvi Shiller during my internship under him, and have significantly improved it in 2015. One of the versions of this algorithm was published in the International Journal of Robotics Research in 2013, and hopefully, being furthered by reinforcement learning, we will be able to submit a revised version of the same. This demo is also a step towards executing a major demo around Diwali, and ensuring that even at high speeds, the vehicle can properly slow down and avoid cluttered environments without diverging too much from its lane, and not stop too much unnecessarily, as most of the autonomous vehicles do in the US and in Europe -- which is not a proper use of the capabilities of autonomous vehicles.

a challenge we have taken on to develop an affordable system, consequently limiting the vehicle's speed. For the product launch, we ma   For the second time now, we have commenced comprehensive testing of our reinforcement learning-based ADAS (Advanced Driver Assistance System) and autonomous driving software on uncontrolled roads in India. Back in 2017/18, we had previously demonstrated the effectiveness of the RL-first approach to autonomous driving on the unpredictable roads of India. Currently in the prototyping stage, the software being showcased in this demo will enable ADAS, lane-keeping, and other L2 autonomous driving features on roads as unstructured as those found in India, all without the need for HD Maps. In this demonstration, the vehicle's core lane-keeping operation relies solely on a single forward-facing GigE-IP camera, out of the six cameras mounted on the roof-top. Our lane detection and generation software, which has been demonstrated multiple times since 2017, enables autonomous vehicles to both detect and generate lane boundaries on-the-fly, actively perceiving the drivable region. A reinforcement learning-based planning and behavior generation software then assumes control of the vehicle. Depth perception is achieved using only a single camera,y opt to utilize a stereo camera system.

This demo showcases our autonomous driving software in the prototyping stage. When productized at Swaayatt Robots, it will enable (i) campus autonomous driving, (ii) autonomous forward and reverse parking with near end-effector constraints satisfaction, (iii) execution of tight S-Curves, and (iv) U-Turns, with intelligence to decide which spot to park at, based on the clutter and the contextual understanding of its environment. In this demo the environment is divided into 3-zones, each zone having some drive lanes and some park-lanes. The core motion planning software plans an optimized trajectory to meet the end-effector constraints and has to decide on the fly which divider passageway is to be selected to transition to the next zone. The planning is online, as opposed to off-line/global planning approach adopted by the industry. Next we would be blending this with highly optimized #reinforcementlearning (RL) based decision-making framework, using Semi Markov Decision Processes (S-MDPs), that would automatically be able to decide whether to stop or not for other vehicles, and when to initiate reverse direction navigation depending on the contextual understanding of the environment. One of my core focuses since undergrad is autonomous navigation in unknown and unseen environments via reinforcement learning. Thus, the learned RL-agents' policies will be independent of the environment / simulation data it is trained in, and would be able to work in any campus environment, with stochastic localization against high-fidelity maps (like GPS localization) in such environments. Further, to scale-up last mile autonomy, in this demo we also show the wire-frame high-fidelity map of the environment, which is useful for such last mile #autonomousmobility tasks.

Autonomous driving, executing U-Turns at high-speeds, i.e., near-drift speeds, and lane-change operations, via reinforcement learning. This video demonstrates Swaayatt Robots (स्वायत्त रोबोट्स) autonomous driving technology enabling (i) high-speed maneuvers, (ii) lane-keeping, and (iii) lane-shift operations. The core navigation policy for the RL agent, which is executed by our autonomous vehicle, was majorly learned in simulations and transferred to the real-world. The vehicle entered the U-Turn at 45 KM/H (which is near-drift speed given the tight-turn and the vehicle's dynamics) and executed it precisely, and thereafter performed a lane switching maneuver to reach its target parking stop. This demo was also performed to machine learn and iteratively-calibrate our system, for the 100 KM/H autonomous driving (which we will be executing within the next 7 months), where the vehicle will have to execute many such high-speed turns on mountainous terrain.

Swaayatt Robots' Autonomous vehicle was featured on Zee Hindustan news channel . In this video, CEO Mr. Sanjeev Sharma is demonstrating the autonomous driving of their vehicle.