CBR talks to the head of IT at Lotus Renault Formula One team about the complex tech infrastructure needed to support a constantly moving team, whether cloud computing is right for them and what happens if it all goes wrong
Let’s start with a bit of history of the Lotus Renault team.
We’ve been in existence for 30 years this year. We started as Tolman, became Benetton, Renault and now it’s Lotus Renault. We’re based in Enstone, which is just north of Oxford. We employ around 520 people, we have our own wind tunnel on site and a supercomputer that we use for computational fluid dynamics, so it’s like a virtual wind tunnel.
So what about the basics of the IT department?
There are about 40 people in the department, just under half of those are developers so we do a lot of software development. Either we write the links and interfaces between our CAD, manufacturing systems, finance system and the trackside systems or we write the applications that are used at the tracks, our own strategy system that takes all of the parameters such as weather, tyre performance, competitor analysis, and brings it all together for the strategist, who can run thousands of race simulations in a couple of minutes so you can quickly get an idea of what the best strategy is.
We’ve also started doing work with a statistical engine this year, so we can ask for all the races where it was raining and we were using a particular type of break and it can pull all that information together.
How much data is created over the course of a Formula One race weekend?
It’s about 25GB. We used to capture one file per lap, so we had a good idea of how big each file was. Now with the Common Electronics Box the whole race is one file. So where we used to be able to stream the data back in real-time – about 15 seconds after the car crossed the line we had the data back at the factory – now we have to wait to the end of the race before we can get that file, so we are kind of restricted in terms of what we can do.
What about using live data to affect race strategy and so on?
We put a lot of emphasis on real-time analysis at the track. Depending on which circuit you’re at you generally have around 90 seconds per lap and that’s how long you have to make a decision. Sometimes the driver is at the second-last corner and you have to decide whether to bring them in, for example.
What do you take to the track for a race weekend?
We always used to take everything that was needed by the team to run at the track. Now we are in a position where we make sure we take everything that is critical to the running of the car. Because our WAN links is so reliable now – we have a 10meg MPLS that’s supplied by the same company at every race – we don’t have to deal with the local providers. This is our fifth season of using it and it’s had 100% uptime in that time so we’re more comfortable with leaving certain things behind.
We have two racks of equipment: storage from NetApp, and virtualised servers at the track.
Can you go through the process of getting ready for a race weekend?
We arrive at an empty garage, we have to put everything in place. In Europe everything is trucked to the circuit and we wheel out the two server racks and stick them in the back of the garage and run the network around the garage to the various people there, such as the gearbox team and the engine fitters.
There are two racks at either side of the garage that replicate each other, one for each car. We also may have to do software upgrades.
With races we have to fly to it all goes into packing crates in aeroplanes and you run the risk of things travelling and not getting there in a working condition. In Singapore for example, when the storage left the factory it was fine but wouldn’t work at the track. We and NetApp worked through the night – 30 hours in a row – and got it up and running before the engineers arrived.
The equipment has to work in a variety of weather conditions, such as Malaysia where it’s humid, Barcelona for winter testing when it’s really cold or Bahrain where it’s very dry. We’ve got two very nice data centres back at the factory but it’s really tough out at the track so for that reason we tend to keep the kit for two to three years before refreshing.
What happens during a race if it all breaks down?
From the Wednesday before a race to the Sunday night once it’s finished we need 100% availability, so we go for the tier one vendors: Cisco, HP, Symantec, NetApp and so on. But it does happen: At Monaco a few years ago we had five servers fail on us. We had enough spares that we were able to move things around and not compromise the car.
It’s not that the car would not be able to run, we’d be able to get it out on the track, but it’s more that they wouldn’t have the information they need to steal a couple of places at pit stops for example. You need the telemetry coming back to the engineers and strategists so they can make decisions about whether our driver should push on, hold back, pit now or wait. I don’t think you could win a race without that information.
And what about your mobile workers?
Of the 520 people about 40% of them are considered mobile workers, so they regularly work away from the factory. We provide laptops for them and they have to be able to take a subset of the data with them so we have all the usual concerns over data security and IP protection.
Last year we installed Symantec Endpoint Encryption on the machines, which was the first time we’d had any encryption on them. Not because we didn’t know there was an issue but because of the performance overhead you got by adding encryption to the machines. We fire up the cars with laptops. If it blue-screens and take seven minutes to boot up you’re not going to be able to start the car in time.
So we had to compromise and accept the risk of not encrypting the machines. But the new encryption technology adds negligible time to the boot-up process.
I really want to get to the stage where we can not worry about the endpoint, and anyone can use any device. The only way I can see that happening is virtualisation. So either you have a personal device and I put a company VM on it or we give them a personal VM on a company-owned machine. You have to be able to separate those two things.
Is that likely any time soon?
I think it’s three years away. We’re about to refresh the clients now and we haven’t become mature enough with VDI to take a chance on it. So we’re replacing like-for-like now, which gives us three years to build our virtual environment.
Are you looking at anything like cloud computing for storage of race data and so on?
The cloud would make sense for storing data because at the moment we have to push it all the way back to the UK. But at the moment we have too many concerns about the security of that data and where it is, where the data centres are. We’re just not comfortable with that yet. We do use some cloud services, such as email security.
With the tech you develop in-house, have you looked at licensing it out?
We’re beginning to. In the past we’ve always wanted to keep it ourselves and if it’s anything that gives us a competitive advantage we wouldn’t. But we have developed a number of systems that could be used in other industries and we have started to look at revenue opportunities like that.
Earlier this year CBR was invited to the Norfolk headquarters of Team Lotus. You can read all about the trip here.