This demo showcases autonomous driving in the wild, where our autonomous vehicle was tasked with avoiding generic static and dynamic obstacles in its path -- with the margin of error being very low. At some points, our autonomous vehicle had few tens of centimetre of margin, from the impending obstacles, with very-rough and bumpy terrain ahead of it. For a significant portion of the navigation, the vehicle had to negotiate bi-directional traffic on a single-lane off-road -- although we developed novel motion planning and decision making frameworks to enable this, around January we will showcase this capability at very high-speeds, via a multi-agent intent analysis and bi-directional negotiation policy, learned via deep unsupervised learning + reinforcement learning. This demo made use of very sparse stochastic HD maps, for stochastic projection of boundary information. In the coming weeks and months we will showcase autonomous driving without any HD map off-roads, showcasing the agent building the cost-map of the navigable region via unsupervised deep learning, including building an inherent understanding of the boundaries of the environment, utilizing only GPS-nodes as waypoints. Our research in off-roads autonomous driving in general focuses on achieving the end-goal of enabling autonomous vehicles traverse any terrain without using HD maps, learning a driving policy via unsupervised machine learning and reinforcement learning -- pushing the boundary of what is considered possible in autonomous driving. This will eventually trickle down to on-road autonomous driving, where we will showcase elimination of the key perception algorithms altogether -- road detection, lane detection, terrain-map, and perhaps obstacles detection as well. In the pursuit of achieving Level-5 autonomy we will be doing a 100 KM/H autonomous driving demo, showcasing: (i) Generic bi-directional negotiation at high-speeds on a single lane road, (ii) over-taking and ability to abort like human drivers do, and (iii) high-speed navigation without any maps. As a first step in this endeavor we will be executing a similar demo on a single lane on-road environment at high-speeds, where often to avoid the on-coming obstacle, the vehicle will have to drift off-roads -- utilizing the stack we have built for generic off-roads autonomous driving, which will be soon scaled with unsupervised learning.

In this demo, our vehicle can be seen traversing through dynamic, unpredictable, and tight obstacles while also ensuring soft constraints of its lane and the environment.

Our research at Swaayatt Robots (स्वायत्त रोबोट्स) in several different fronts of theoretical computer science and applied mathematics, including, but not limited to #reinforcementlearning, transfer learning, #deeplearning, #machinelearning, motion planning, SLAM and mapping, computer vision, stochastic processes, manifold alignment, that we have been doing over the years, and demonstrated through 60+ demos over the past few years, is now taking a definitive shape to enable sustainable autonomous driving at scale.

Next will showcase our R&D in enabling mapping at a very large scale via both contextual and symbolic transfer of knowledge across the maps, and also showcase generalized obstacles avoidance off-roads, first via using perception algorithms, and then by eliminating several of the perception algorithms, without any HD maps, via the novel unsupervised learning based decision making framework we have been working on.

We expect to open autonomous driving, and #autonomousvehicles experience to the public in India in the month of October. This will be historic in the context of autonomous driving. Very soon we will execute high-speed autonomous driving on a mountainous terrain as well.
 

Navigating at night at high-speeds (max speed during the demo 64 KM/H), safely executing lane keeping, merging into a traffic intersection, and safely executing round-about navigation policy.

While the roads over here were a bit well-paved, given our autonomous lane detection and generation algorithm, which is being scaled, for both day and night operations, that automatically generates lane markers on-the-fly in the absence of lanes markers in the visual-sensory data, we are able to tackle both structured- and unstructured-roads (as we had demoed in the past). Now this is being scaled for state-wide and country-wide operations.

Earlier we had demoed off-roads autonomous driving, via #reinforcementlearning, and when that research is merged with our on-roads autonomous driving in the coming weeks, we will have scalable #autonomousvehicles capable of negotiating any kind of roads and environmental situations.

Next we will be showcasing autonomous driving at night through moderate traffic conditions of the Indian cities, and very-soon during the day-time as well.

It can be seen that our autonomous driving software, which uses reinforcement learning, enabled our autonomous vehicle to navigate through very-tight regions in the campus. Executing such tight turns required fundamentally innovating the controller algorithm on the fly, within a few hours -- I had to use my several years of mobile robotics and motion planning experience to get this done rightly. Our vehicle had driven all the way from Bhopal to Mussoorie, through rain, dust, and fog, around 1100 KM one way trip. As a result, on the 20th, our braking system and many of the cameras failed to function. Thus, we were not able to showcase our full capabilities on the 20th. However, after much effort, we were able to activate our braking system on the 21st of March, and again did the demo, that is being showcased here. This was an unknown campus to us, and with a few hours of fine-tuning, we were able to transfer our campus autonomous driving software to function at the previously unseen LBSNAA campus. This again highlights the readiness of our campus autonomous driving software to execute autonomous driving anywhere with minimal efforts.

Autonomous driving off-roads at high-speeds, a one of its kind of a demo, handling such levels of uncertainty and unstructured off-road.

The core motion planning and decision making in this demo is being governed by a reinforcement learning agent. The vehicle reached a top speed of 44 KM/H, which is near the limit of humans driving safely on this road, without risking drift or side-skids at turns.

At Swaayatt Robots (स्वायत्त रोबोट्स) we have initiated a research project, with the long term goal being, at least for off-roads, to enable the autonomous vehicle make decisions about motion and behavior without relying on explicit computation of the various perceptual features, for example, localization against HFMs or detection of delimiters or obstacles.

This demo showcases a glimpse of the classical variant of the underlying research put into action. In the end, in a few months hereafter, this novel decision making and motion planning algorithmic framework will change the off-roads #autonomousdriving paradigm and how #autonomousvehicles compute actions or make navigation decisions.

The deep #reinforcementlearning variant of this algorithmic framework will completely eradicate the need for explicit perception algorithms, and autonomous vehicles will be able to make a gist of their environment and take behavioral actions, without explicitly detecting road boundaries or delimiters as well as obstacles and computing their intentions. This research will then be utilized on-roads to achieve scalable Level-5 autonomous driving.

This demo used explicit obstacles detection, which worked robustly on-roads, as well as two LiDARs and cameras.

Autonomous driving through generalized traffic with autonomous parking maneuver via reinforcement learning-based driving policy. We are going to begin executing autonomous driving on urban roads, i.e., city roads with unruly traffic, in India starting March/April, and this demo is a part of a series of demos and trials we will be doing along the way, to both achieve autonomous urban driving and 100 KM/H L-4 autonomous driving in India. In this demo, our autonomous vehicle is tasked with negotiating and avoiding generalized uncontrolled traffic with an end goal of parking at a designated spot with near-end-effector constraints. The complexity of this task is humongous, given the unpredictable pattern of the traffic and pedestrians in an open environment -- there are no traffic rules that can provide insights into the behavior of pedestrians or other vehicles. This was achieved through an autonomous driving policy learned via reinforcement learning. Earlier in 2017 as well, we had demoed a multi-agent reinforcement learning-based framework to perform autonomous navigation through stochastic tight roads in India with generalized traffic. The present demo and the earlier 2017 demo make us the only autonomous vehicles startup to put a full-fledged autonomous SUV on Indian roads.

Autonomous driving off-roads with very thick fog! This capability makes us the 3rd startup/company in the world to have both on- and off-roads autonomous driving capabilities, after Wayve AI (2018) and Oshkosh Defense. Furthermore, we are the only startup in the world to demo autonomous driving capabilities in India and actively pursuing Level-5! The ongoing DARPA RACER program's objective is to drive off-roads at human-driven speeds. We acknowledge the work, specifically by University of Washington's team, where they are building an explicit cost-map to drive off-roads. Our work is more unsupervised. We are perfecting the off-road autonomous driving policy, including optimal speed control over off-roads, without any prior map of the environment, and without any explicit algorithm for processing of the visual sensory information, and we should be able to demo this capability in the coming weeks in the form of a PoC. This demo utilized one VLP-32C LiDAR from Velodyne. Out of the 25+ trials we did, there was one point where manual intervention was felt necessary, as this was a single-lane road with bi-directional traffic. This kind of negotiation is an active area of research at Swaayatt Robots (स्वायत्त रोबोट्स), and we are learning a policy to solve this problem.

Autonomous driving navigating through tight obstacles validating two of the motion planning algorithms that we are furthering at Swaayatt Robots, and for which will be submitting three papers, two of those perhaps in Robotics Science and Systems conference, and filing provisional patent for the one. This demo required integration of active and passive perception algorithmic pipelines, and by the end of 2022, we will finish most of the campus autonomous driving software, with parking feature, which will go on and become the first commercial offering from Swaayatt. In this demo, one of the motion planning algorithms, that is being furthered with reinforcement learning, is the one I originally invented when I was an undergrad at IIT Roorkee. It is the first time we are testing it on autonomous vehicles as it guarantees safety mathematically for mobile robots exhibiting kino-dynamics, such as a campus vehicle navigating at manageable speeds. The second is an algorithm that I developed with Zvi Shiller during my internship under him, and have significantly improved it in 2015. One of the versions of this algorithm was published in the International Journal of Robotics Research in 2013, and hopefully, being furthered by reinforcement learning, we will be able to submit a revised version of the same. This demo is also a step towards executing a major demo around Diwali, and ensuring that even at high speeds, the vehicle can properly slow down and avoid cluttered environments without diverging too much from its lane, and not stop too much unnecessarily, as most of the autonomous vehicles do in the US and in Europe -- which is not a proper use of the capabilities of autonomous vehicles.

At Swaayatt Robots, we have initiated autonomous driving trials on city roads in India during the nighttime, and our plan is to expand these trials to daytime operations by Q2-2023, even amidst the chaotic traffic. During these trials, we will be showcasing the following: 1. The decision-making capabilities of our reinforcement learning-based autonomous driving policies. 2. Navigation capabilities with stochastic localization against high-fidelity maps. 3. High-precision localization against high-fidelity maps for last-mile autonomy. 4. Perception techniques using both LiDAR and non-LiDAR based approaches. These demonstrations represent significant steps in advancing our autonomous driving technologies. Furthermore, our autonomous driving trials on city roads in India serve as a testament to the adaptability of our technology in diverse and complex real-world environments. Navigating through the bustling streets, our autonomous vehicles showcase their ability to respond to unpredictable scenarios, ensuring safety and efficiency even in challenging traffic conditions. As we transition to daytime trials, our commitment to pushing the boundaries of autonomous driving remains unwavering. We envision a future where our technology not only enhances convenience and safety on the roads but also contributes to the evolution of urban mobility, making transportation more sustainable and accessible for all. With continuous innovation and rigorous testing, we are excited to bring the promise of autonomous driving closer to reality.

a challenge we have taken on to develop an affordable system, consequently limiting the vehicle's speed. For the product launch, we ma   For the second time now, we have commenced comprehensive testing of our reinforcement learning-based ADAS (Advanced Driver Assistance System) and autonomous driving software on uncontrolled roads in India. Back in 2017/18, we had previously demonstrated the effectiveness of the RL-first approach to autonomous driving on the unpredictable roads of India. Currently in the prototyping stage, the software being showcased in this demo will enable ADAS, lane-keeping, and other L2 autonomous driving features on roads as unstructured as those found in India, all without the need for HD Maps. In this demonstration, the vehicle's core lane-keeping operation relies solely on a single forward-facing GigE-IP camera, out of the six cameras mounted on the roof-top. Our lane detection and generation software, which has been demonstrated multiple times since 2017, enables autonomous vehicles to both detect and generate lane boundaries on-the-fly, actively perceiving the drivable region. A reinforcement learning-based planning and behavior generation software then assumes control of the vehicle. Depth perception is achieved using only a single camera,y opt to utilize a stereo camera system.

This demo showcases our autonomous driving software in the prototyping stage. When productized at Swaayatt Robots, it will enable (i) campus autonomous driving, (ii) autonomous forward and reverse parking with near end-effector constraints satisfaction, (iii) execution of tight S-Curves, and (iv) U-Turns, with intelligence to decide which spot to park at, based on the clutter and the contextual understanding of its environment. In this demo the environment is divided into 3-zones, each zone having some drive lanes and some park-lanes. The core motion planning software plans an optimized trajectory to meet the end-effector constraints and has to decide on the fly which divider passageway is to be selected to transition to the next zone. The planning is online, as opposed to off-line/global planning approach adopted by the industry. Next we would be blending this with highly optimized #reinforcementlearning (RL) based decision-making framework, using Semi Markov Decision Processes (S-MDPs), that would automatically be able to decide whether to stop or not for other vehicles, and when to initiate reverse direction navigation depending on the contextual understanding of the environment. One of my core focuses since undergrad is autonomous navigation in unknown and unseen environments via reinforcement learning. Thus, the learned RL-agents' policies will be independent of the environment / simulation data it is trained in, and would be able to work in any campus environment, with stochastic localization against high-fidelity maps (like GPS localization) in such environments. Further, to scale-up last mile autonomy, in this demo we also show the wire-frame high-fidelity map of the environment, which is useful for such last mile #autonomousmobility tasks.

Autonomous driving, executing U-Turns at high-speeds, i.e., near-drift speeds, and lane-change operations, via reinforcement learning. This video demonstrates Swaayatt Robots (स्वायत्त रोबोट्स) autonomous driving technology enabling (i) high-speed maneuvers, (ii) lane-keeping, and (iii) lane-shift operations. The core navigation policy for the RL agent, which is executed by our autonomous vehicle, was majorly learned in simulations and transferred to the real-world. The vehicle entered the U-Turn at 45 KM/H (which is near-drift speed given the tight-turn and the vehicle's dynamics) and executed it precisely, and thereafter performed a lane switching maneuver to reach its target parking stop. This demo was also performed to machine learn and iteratively-calibrate our system, for the 100 KM/H autonomous driving (which we will be executing within the next 7 months), where the vehicle will have to execute many such high-speed turns on mountainous terrain.

Swaayatt Robots' Autonomous vehicle was featured on Zee Hindustan news channel . In this video, CEO Mr. Sanjeev Sharma is demonstrating the autonomous driving of their vehicle.