Tackling the 3D Simulation League: an interview with Klaus Dorer and Stefan Glaser

A screenshot from the brand new simulator that shall be trialled for a particular problem at RoboCup2025.

The annual RoboCup occasion, the place groups collect from throughout the globe to participate in competitions throughout quite a few leagues, will this yr happen in Brazil, from 15-21 July. Upfront of kick-off, we spoke to 2 members of the RoboCup Soccer 3D Simulation League: Govt Committee Member Klaus Dorer, and Stefan Glaser, who’s on the Upkeep Committee and who has been not too long ago creating a brand new simulator for the League.

Might begin by simply giving us a fast introduction to the Simulation League?
Klaus Dorer: There are two Simulation Leagues in Soccer: the 2D Simulation League and the 3D Simulation League. The 2D Simulation League, because the title suggests, is a flat league the place the gamers and ball are simulated with simplified physics and the primary focus is on group technique. The 3D Simulation League is way nearer to actual robots; it simulates 11 versus 11 Nao robots. The extent of management is like with actual robots, the place you progress every motor of the legs and the arms and so forth to realize motion.

I perceive that you’ve got been engaged on a brand new simulator for the 3D League. What was the thought behind this new simulator?
Klaus: The intention is to convey us nearer to the {hardware} leagues in order that the simulator will be extra helpful. The present simulator that we use within the 3D Simulation League is named SimSpark. It was created within the early 2000s with the intention of constructing it potential to play 11 vs 11 gamers. With the {hardware} constraints of that point, there needed to be some compromises on the physics to have the ability to simulate 22 gamers on the identical time. So the simulation is bodily considerably practical, however not within the sense that it’s straightforward to transpose it to an actual Nao robotic.

Stefan Glaser: The thought for creating a brand new simulator has been round for a number of years. SimSpark is a really highly effective simulation framework. The bottom framework is area unbiased (not soccer particular) and particular simulations are realized through plugins. It helps a number of physics engines within the backend and supplies a versatile scripting interface for configuration and diversifications of the simulation. Nevertheless, all this flexibility comes with the value of complexity. Along with that, SimSpark makes use of customized robotic mannequin specs and communication protocols, limiting the quantity of accessible robotic fashions and requiring groups to develop customized communication layers just for speaking with SimSpark. Because of this, SimSpark has not been extensively adopted within the RoboCup group.

With the brand new simulator, I wish to tackle these two main points: complexity and standardization. Within the ML group, the MuJoCo physics engine has turn out to be a very fashionable alternative for studying environments after Google DeepMind acquired it and launched it open supply. Its requirements for world and robotic mannequin specs are extensively adopted in the neighborhood and there exist loads of ready-to-use robotic mannequin specs for all kinds of digital in addition to real-world robots. In the midst of final yr, they (MuJoCo) added a function which lets you manipulate the world illustration throughout simulation (including and eradicating objects to / from the simulation whereas preserving the simulation state). That is one important requirement now we have within the simulation league, the place we begin with an empty area after which the brokers join on demand and type the groups. When this function has been added, I made a decision to make a step ahead and attempt to implement a brand new simulator for the 3D Simulation League primarily based on MuJoCo. Initially, I wished to start out growth in C/C++ to realize most efficiency, however then determined to start out in Python to scale back complexity and make it extra accessible for different builders. I began growth on Easter Monday so it’s not even three months previous!

I believe it could be helpful to clarify a bit bit extra in regards to the setup of our league and the necessities of the simulator. If we take the FIFA sport (in your favourite gaming system) for instance, there may be one simulation taking place which simulates 22 gamers and the choice making is a part of the simulation having full entry to the state of the world. Within the 3D Simulation League now we have two groups with 11 robots on the sector, however we even have 22 particular person agent softwares that are linked to the simulation server, every controlling one single robotic. Every linked agent solely receives sensor data associated to their robotic within the simulation. They’re additionally solely allowed to speak through the server – there isn’t a direct communication between the brokers allowed in Simulation League. So now we have a basic setup the place the simulation server has to have the ability to settle for as much as 22 connections and handle the scenario there. This performance has been the most important focus for me for the final couple of months and this half is already working properly. Groups can join their brokers, which is able to obtain sensor data and may actuate joints of the robotic within the simulation and so forth. They’re additionally capable of choose completely different robotic fashions in the event that they like.

An illustration of the simulator set-up.

Presumably the brand new simulator has a greater illustration of the physics of an actual robotic.
Klaus: Precisely. For instance, how the motors are managed is now a bit completely different and far nearer to actual robots. So once I did my first experiments, I noticed the robotic collapse and I assumed it was precisely how an actual robotic would collapse! In SimSpark we additionally had falling robots however the motor management within the new simulator is completely different. Now you possibly can management the motors by velocity, by power, by place, which is way more versatile – it’s nearer to what we all know from actual robots.

I believe that, not less than initially, it is going to be harder for the Simulation League groups to get the robots to do what they need them to do, as a result of it’s extra practical. For instance, in SimSpark the bottom contact was way more forgiving. So for those who step onerous on the bottom, you don’t fall instantly with a SimSpark robotic however with a MuJoCo robotic this shall be way more practical. Certainly, in actual robots floor contact is considerably much less forgiving.

I had a query in regards to the imaginative and prescient facet – how do the person brokers “see” the place of the opposite brokers on the sector?
Stefan: We simulate a digital imaginative and prescient pipeline on the server aspect. You will have a restricted area of view of ±60° horizontally and vertically. Inside that area of view you’ll detect the pinnacle, the arms, the ft of different gamers, or the ball, for instance, or completely different options of the sector. Just like frequent real-world imaginative and prescient pipelines, every detection consists of a label, a course vector and the space data. The knowledge has some noise on it like actual robots have, too, however groups don’t have to course of digital camera photos. They get the detections straight from the simulation server.

We’ve beforehand had a dialogue about transferring in the direction of getting digital camera photos of the simulation to combine into the imaginative and prescient pipeline on the agent aspect. This was by no means actually practical in SimSpark with the implementation we had there. Nevertheless, it needs to be potential with MuJoCo. Nevertheless, for the primary model, I used the identical method the normal simulator handled the imaginative and prescient. Which means that groups don’t want to coach a imaginative and prescient mannequin, and don’t have to deal with digital camera photos to get began. This reduces the load considerably and likewise shifts the main focus of the issue in the direction of movement and choice making.

Will the simulator be used at RoboCup 2025?
Stefan: We plan to have a problem with a brand new simulator and I’ll attempt to present some demo video games. In the intervening time it’s probably not in a state the place you possibly can play an entire competitors.

Klaus: That’s normally how we proceed with new simulators. We’d not transfer from one to the opposite with none intermediate step. We may have a problem this yr at RoboCup 2025 with the brand new MuJoCo simulator the place every collaborating group will attempt to educate the robotic to kick so far as potential. So, we won’t be taking part in an entire sport, we received’t have a number of robots, only a single robotic stepping in entrance of the ball and kicking the ball. That’s the technical problem for this yr. Groups will get an concept of how the simulator works, and we’ll get an concept of what must be modified within the simulator to proceed.

This new problem shall be voluntary, so we aren’t positive what number of groups will take part. Our group (MagmaOffenburg) will definitely participate. It is going to be attention-grabbing to see how properly the groups carry out as a result of nobody is aware of how far a great kick is on this simulator. It’s a bit like in Components One when the foundations change and nobody is aware of which group would be the main group.

Do you could have an concept of how a lot adaptation groups should make if and while you transfer to the brand new simulator for the total matches?
Stefan: As a long-term member of 3D Simulation League, I do know the previous simulator SimSpark fairly properly, and know the protocols concerned and the way the processes work. So the primary model of the brand new simulator is designed to make use of the identical primary protocol, the identical sensor data, and so forth. The thought is that the groups can use the brand new simulator with minimal effort in adapting their present agent software program. So they need to be capable of get began fairly quick.

Though, when designing a brand new platform, I wish to take the chance to make a step ahead when it comes to protocols, as a result of I additionally wish to combine different Leagues within the long-term. They normally produce other management mechanisms, they usually don’t use the identical protocol that’s outstanding in 3D Simulation. Subsequently there must be some flexibility sooner or later. However for the primary model, the thought was to get the Simulation League prepared with minimal effort.

Klaus: The large concept is that this isn’t simply used within the 3D Simulation league, but in addition as a helpful simulator for the Humanoid League and likewise for the Customary Platform League (SPL). So if that seems to be true, then it is going to be fully profitable. For the Kick Problem this yr, for instance, we use a T1 robotic that could be a Humanoid League robotic.

Might you say one thing about this simulation to actual world (Sim2Real) facet?
Stefan: We’d prefer it to be potential for the motions and behaviors within the simulator to be ported to actual robots. From my viewpoint, it could be helpful the opposite method spherical too.

We, as a Simulation League, normally develop for the Simulation League and due to this fact wish to get the behaviors operating on an actual robotic. However the {hardware} groups normally have an identical problem once they wish to take a look at high-level choice making. They may have two to 5 robots on the sector, and in the event that they wish to play a high-level decision-making match and prepare in that regard, they all the time need to deploy loads of robots. If additionally they wish to have an opponent, they need to double the quantity of robots with a purpose to play a sport to see how the technique would prove. The Sim2Real facet can also be attention-grabbing for these groups, as a result of they need to be capable of take what they deployed on the actual robotic and it also needs to work within the simulation. They will then use the simulation to coach high-level expertise like group play, participant positioning and so forth, which is a difficult facet for the actual robotic leagues like SPL or the Humanoid Leagues.

Klaus: And the rationale we all know it’s because now we have a group within the Simulation League and now we have a group within the Humanoid League. In order that’s one more reason why we’re eager to convey this stuff nearer collectively.

How does the refereeing work within the Simulation League?
Klaus: A pleasant factor about Simulation Leagues is that there’s a program which is aware of the actual state of the world so we will construct within the referee contained in the simulator and it’ll not fail. For issues like offside, whether or not the ball handed the objective line, that’s fail protected. All of the referee choices are taken by the system itself. We’ve got a human referee however they by no means have to intervene. Nevertheless, there are conditions the place we wish synthetic intelligence to play a job. This isn’t at the moment the case in SimSpark as a result of the foundations are all onerous coded. We’ve got loads of fouls which can be debatable. For instance, there are lots of fouls that groups agree shouldn’t have been a foul, and different fouls that aren’t referred to as that ought to have been. It might be a pleasant AI studying job to get some conditions judged by human referees after which prepare an AI mannequin to raised decide the foundations for what’s a foul and what isn’t a foul. However that is at the moment not the case.

Stefan: On the brand new simulator I’m not that far into the event that I’ve applied the automated referee but. I’ve some primary algorithm which progress the sport as such, however judging fouls and deciding on particular conditions isn’t but applied within the new simulator.

What are the following steps for creating the simulator?
Stefan: One of many subsequent main steps shall be to refine the physics simulation. As an example, regardless that there exists a ball within the simulation, it’s not but rather well refined. There are loads of physics parameters which now we have to resolve on to replicate the actual world pretty much as good as potential. This may possible require a collection of experiments with a purpose to get to the right values for numerous features. On this facet I’m hoping for some engagement of the group, as it’s a nice analysis alternative and I personally would like the group to resolve on a generally accepted parameter set primarily based on a degree of proof that I can’t simply present all on my own. So in case somebody is excited about refining the physics of the simulation such that it greatest displays the actual world, you’re welcome to hitch!

One other main subsequent step would be the growth of the automated referee of the soccer simulation, deciding on fouls, dealing with misbehaving brokers and so forth. Within the first model, foul situations will possible be judged by an knowledgeable system particularly designed for this goal. The simulation league has developed a set of foul situation specs which I plan to adapt. In a second step, I wish to combine and assist the event of AI primarily based foul detection fashions. However yeah, one step after the opposite.

What are you significantly trying ahead to at RoboCup2025?
Klaus: Properly, with our group now we have been vice world champion seven occasions in a row. This yr we’re actually hoping to make it to world champion. We’re very skilled in getting losses in finals and this yr we’re trying ahead to altering that, from a group perspective.

Stefan: I’m going to Brazil with a purpose to promote the simulator, not only for the Simulation League, but in addition throughout the boundaries for the Humanoid Leagues and the SPL Leagues. I believe that this simulator is a good likelihood to convey folks from all of the leagues collectively. I’m significantly within the particular necessities of all of the groups of the completely different leagues. This understanding will assist me tailor the brand new simulator in the direction of their wants. That is considered one of my main highlights for this yr, I might say.

You will discover out extra in regards to the new simulator on the undertaking webpage, and from the documentation.

Klaus Dorer is professor for synthetic intelligence, autonomous programs and software program engineering at Offenburg College, Germany. He’s additionally a member of the Institute for Machine Studying and Analytics IMLA. He has been group chief of the RoboCup simulation league groups magmaFreiburg (since 1999), dwelling programs, magmaFurtwangen and is now group chief of magmaOffenburg since 2009. Since 2014, he has additionally been a part of the humanoid grownup dimension league group Sweaty.

Stefan Glaser is educating assistant for synthetic intelligence and clever autonomous programs on the Offenburg College, Germany. He has been a part of the RoboCup simulation league group magmaOffenburg since 2009 and the RoboCup humanoid grownup dimension league group Sweaty since 2014.