Developing IMU Sensors For Capturing Motion In Sports

Posted on August 4, 2018 by Julian Chua

IMU sensors are pretty useful because when strapped to the right location and given the right context they can provide very insightful information about an athlete’s (or anyone’s) movements. In this post, we are going to look at a couple of options in the market that allows us to skip the hardware development and jump right into the application development. Feel free to skip to the different sections that interest you:

[ Intro To IMUs – mbientlabs – Notch Sensor – Notch Mocap Test – Custom Sensors]

Intro To IMUs

In case this is the first time you are hearing about IMU, here’s a brief intro. IMU stands for Inertial Measurement Unit; it is an electronic device that typically has accelerometers, gyroscopes and magnetometers, and it measures its own acceleration, angular rate (or spin rate) and surrounding magnetic field. IMUs are not only used in sports, in fact, it is used in many consumer electronic devices. Our smartphones for one has IMUs for detecting the orientation of the phone and changing the display to portrait or landscape. The IMUs also allows for functions such as undoing texting errors, a spirit level and motion sensor games. If a user carries the phone with them in their pockets most of their waking hours, it can act as a pedometer counting steps and detect when the user is sedentary. For runners who use running apps to track their runs, IMUs enable some apps to track indoor runs and cadence. Sports Engineering Researchers have used smartphones for tracking wheelchair rugby activities and classifying different sporting activities.

As great as the smartphones are with inbuilt IMU, GPS and processing power to give us real-time analysis, we don’t really want to strap an expensive smartphone onto a football player’s calf to monitor their kicking or tape an iPhone to a tennis racket to measure swing metrics. That’s why companies like Qlipp has developed sensors for tennis or Zepp which has sensors for a number of bat-and-ball or swing type sports. Then there are sensors for rowing, running, surfing, mountain biking and more. There are also different sports equipment that has in-built IMU sensors. Like smart balls (basketball, football, cricket ball etc), smart shoes, smart helmets, smart rackets etc, it could go on and on.

But sometimes we might still not find a sensor product on the market that is right for our sports or health application. So we explore the option of developing something on our own. Fortunately, we don’t necessarily have to start from scratch* because these days there are generic IMU sensor platforms that are designed and built for people who want to develop a sensor for a custom application. They often have the standard 9-DOF (degree of freedom) sensor setup and come with software SDK that allows developers to build their own applications for processing and analysing the data. Let’s look at a couple of options below.

[*when I say scratch, I mean getting sensor boards from SparkFun, Adafruit, Seeedstudio, Tindie etc]

mbientlab

mbientlab successfully launched their first Bluetooth IMU sensor on Kickstarter. They pitched it as a development and production platform for wearables with simple API for iOS and Android. There was some simple soldering required when people bought the first product. I didn’t get one from that campaign but I did get a later updated version which they called MetawearRG. What impressed me when I first got it was the size of it – it’s small and compact and I could use it to build/redesign a smart basketball prototype for a client. Then when I started testing it, I found that their API was really easy to use and I could use their sample iOS app to build a custom app for testing within a (reasonably) short time.

: Smart Basketball Prototype and Watch app for tracking optimal shots

Since then, they have made many other versions of sensors with:

slightly different sensor configurations,
options of coin cell or rechargeable lithium battery,
accessories such as cases, clips or wristbands,
sensor fusion firmware,
cloud services, and
hubs to manage multiple sensors.

: Metawear RG with custom 3d printed sleeve/case (L) and Metamotion (R)

I haven’t had the chance to try everything but I have to say, I have had a good experience using their Metawear and Metamotion sensors to build various proof of concepts and I am still using them for a number of projects. The sensor data can be streamed to your smartphone or logged on the device. In terms of API support, on top of iOS and Android, they have added Python, C, C# and Javascript, so developers can build stuff on various platforms.

Sample/Template Metawear iOS app for testing

Looking at their new website revamp and some recent emails they sent out about new platform developments, they seem to be putting more focus into the allied health space, in particular, measuring range-of-motion (ROM). They are currently beta testing an app called the MetaClinic and it looks like they are using skeleton-tracking the likes of motion capture systems which would probably mean we need to use multiple sensors. That should be interesting.

MetaClinic App by mbientlab

Notch Sensors

Notch also launched on kickstarter, in fact slightly earlier than mbientlabs’ campaign. They had an interesting concept of integrating individual IMUs into custom designed clothing using pockets in discreet locations. Unfortunately, they weren’t successful at that instance. Their initial use case probably wasn’t strong enough. So I guess the founders went back to the drawing board, revamped it all and went with the “motion capture” approach for developers.

Notch sensor with elastic band and clip

With the new design, the shape of the IMU sensor is essentially the same but they have ditched the micro-usb in each IMU for contact pins and made it water-resistant (IP67). They also designed elastic bands of varying lengths with a sensor clip and a user can secure each sensor up to 15 different locations on their body including head, chest, upper arms, wrists, hands, waist, thighs, ankles and feet. So instead of selling individual IMUs, they sell a kit of 6 IMUs with a set of elastic bands, and if a user wants to do a full (body) setup, they will need 3 kits.

The Pioneer Kit: 6 IMUs with charging case and elastic bands with clips.

A quick test and review (for biomechanics)

I had the opportunity to run a short pilot test with one (the pioneer) kit in a biomechanics lab. I used the lower body setup which used all 6 IMUs strapped on my chest, waist, thighs and shins/ankles. In terms of setting up, it was pretty straightforward. After following an initial calibration procedure of all the IMUs in the case, I put on the bands and clipped each IMU to the right location according to the different colours as indicated on the app. The only thing is putting on the bands takes a bit of practice and I had to swing around to check that the bands are not too tight and restricting movement. Even though I don’t have muscly quads, I felt that the bands were somewhat tight and needed adjusting after a while.

Setting up the Notch IMUs for lower body measurements

For testing, I did a simple protocol of walking, stopping and doing 3 squats of varying depths. Then I compared my knee angles measured on the notch and the motion capture system. A few quick things that I took out of the knee angle measurements were:

The angle measured by Notch is the exterior angle while the motion capture system looks at the interior angle. So it needs a quick recalculation before comparison.
Assuming the motion capture system is the more accurate measurement, Notch had a larger error as squats went deeper.
But for walking, the knee angles measured were quite close.

It’s wasn’t a very elaborate test but even from this simple outcome, I can safely say it’s probably not the best tool for accurate joint angle measurements. Although for a quick 3D visual feedback on movements, it might work. Here’s the clip of me doing the test described above (feel free to rotate the video to get different perspectives):

Further to that, I could only download angle data. If I wanted the raw sensor (acceleration and gyro) data, I would need to pay for an extended license that is renewed annually.

In terms of custom development support, they used to have support for iOS but they seem to have taken that off now and only have support for Android which I thought is a bummer. I am guessing they have some issues with getting it right on iOS. Hopefully, it is just temporal and they will resolve it soon. For Android developers, it looks like they have pretty good support and even provides a template app. I have to add that there is a fair bit of fine print I need to agree to before I can get access to their SDK. If I read it right, they basically want a licensing fee for using/commercialising their SDK.

Custom Sensors

Both of the above IMU sensors have similar specifications when it comes to measuring acceleration (using accelerometers) and angular velocity (using gyroscopes). The typical measurement range for accelerometers is +/-16g (that’s 16 times of gravitational acceleration), and for gyroscopes, it’s +/- 2000 degrees per sec. For many applications, this configuration is fine. But there might be some cases where higher acceleration needs to be measured and that goes beyond 16g, like shocks or high impact collisions. Or I might need high-speed rotations to be tracked and 2000 degrees per sec is too low, like measuring the spin of a cricket ball or gridiron football (which can come close to 3600 degrees per sec or 600rpm as demonstrated here by Drew Brees).

Spin rates of a gridiron football during a throw test

As briefly mentioned earlier, hobby electronics stores like SparkFun, Adafruit, or Tindie would be a good place to start when looking for accelerometers and gyroscopes of different specifications. There are also lots of microcontrollers with Bluetooth Low Energy (BLE) built-in that are Arduino compatible so we can program them with the Arduino software. One that I found pretty handy is this one called Blueduino which comes with a Lipo charger add-on (and add-ons are great) and that can be found on Tindie.

The gridiron football sensor prototype using the Blueduino

Final Word

For those who are in research and possibly need Matlab and software support for building custom Matlab programs, definitely check out Sabel Sense sensors (Australia). Else, I reckon the mbientlab sensors would be a great option for starting a custom development. If I get a chance to trial their Metaclinic platform, I will put up another post. Meanwhile, do drop me a message here if you need assistance or advice in any of the options above and feel free to leave a comment if you know of better/different solutions out there. With that, thanks for reading!

A Look at Smart Balls

Posted on July 26, 2014 by Julian Chua

Tracking how fast a ball was kicked or thrown used to be done with an external device – it could be a speed radar or a high speed camera or maybe even a very trained (and experienced) eye. However, in the last 5-6 years, more and more engineers and scientists have tried to put some form of sensors inside the balls to measure linear velocity, spin velocity, spin axis. This has mostly been made possible with advanced developments in microelectromechanical sensors (MEMS), where accuracy and measurement range has increased significantly (while still keeping the small form factor). Another 2 tech contributions that helped keep the sensors (more permanently) in the balls are wireless connectivity (Bluetooth or Wifi) with the micro-controllers and wireless charging.

Smart Ball Construction

Although the electronics is key to measuring movement signals and processing, there is still the very important task of holding those components (sensors + micro-controller + wireless modules + battery) inside the ball. Let’s call all those components the core. So while designing a method to secure the core within the ball, one has to consider the weight and position of the core and how it affects the centre of mass of the ball. The method has to be robust enough since the ball will take lots of impacts as it’s kicked or thrown or bounced. The method of securing the core will also affect or determine how the ball is constructed. Here’s a look at some of the different type of “smart” balls and their construction:

Smart Basketball: 94Fifty

Image from their patent file

The way that the 9DOF sensor is built into the 94Fifty ball is rather unique (thus the patent). According to their patent application, there is an inner cavity on the surface of the inside of the ball, which is purposed for a casing to house the electronic components (core). The casing is built with a flexible material such that the walls can flex with the pressure difference between the inside of the bladder and the inside of the housing. The patent application also mentions providing access for battery charging but that was probably the early version. The new version is built with Bluetooth connectivity and wireless charging.

The ball is constructed according to the official size and weight which is 29.5 inches (749.3mm) and 22 ounces (623.7g). So with the extra weight added from the core, the designers made adjustments to the enclosure material so that the overall weight is close to the standard weight, and more importantly, the weight distribution is compensated so it spins like a standard ball. For example, if the core is positioned at the top of the ball (see image above), and the valve is placed 180 degrees from the core, the extra weight would be added around the valve until the balance is achieved.

Smart Soccer ball: adidas micoach

adidas miCoach Smartball

adidas’ smart ball is designed with its core positioned within the ball and held there by what looks like 12 sets of supports. The core is positioned or suspended right in the centre of the ball, and the supports are meant to be rigid so that the core is always in the dead centre. There doesn’t seem to be any patent related to the method of supporting the core but there was a patent with regards to the electrical wiring within the ball. The patent basically describes how the wiring is arranged along the bladder wall to interconnect two electronic devices. It also mentions that the electronic components are arranged in such as way that the ball is balanced and doesn’t affect playing properties of the ball. According to the adidas page, the core consists of only a tri-axial accelerometer. There is also wireless charging with their custom induction-charging stand. The induction coils would likely be placed along the bladder wall instead of in the core.

Smart Cricket ball

The Sportzedge group at RMIT developed an instrumented cricket ball for measuring the spin rate and calculating the position and movement of the spin axis (link to the conference paper). Due to the high spin rates of wrist spinners (up to 42 rps or 15,120 deg/s), typical off the shelf gyroscope sensors can’t manage that measurement range. What this smart cricket ball has are three high-speed gyros that can measure +/- 20,000 deg/s, one for each axis. This ball is not built in the typical manufacturing process. In order to house the electronics, meet weight requirements, and keep it balanced, 2 solid halves of the ball were designed and CNC machined from the material Ureol or RenShape® BM 5460 which had the right density and hardness. Eight holes within the ball allowed for additional masses to be inserted to balance the ball. According to the paper, this design is an initial prototype and it is still not robust enough to be hit by a cricket bat. But it is fully capable of measuring spin rates during fast bowling. Subsequent versions will be more sturdy and also include wireless charging.

Instrumented cricket ball (source: Fig 1 of the research paper)

Smart Oval Ball

Smart AFL ball

The same team that built the smart cricket ball also developed a smart AFL ball to assess angular flight dynamics and precision of kick execution. The same electronics (high-speed gyros) that were built into the smart cricket ball was also incorporated into this smart oval ball. The main difference is, this oval ball is made with two bladders that sandwich the core electronics, keeping them right in the middle of the ball. The bladders were inflated simultaneously to ensure a more even distribution of pressure. It was noted in their paper that the advantage of using an inflatable bladder (instead of replacing it with expanded polystyrene beads) is that it allows for realistic kicking whereas the foam beads will absorb too much energy thus dampening the performance. Other than the smart AFL ball, a recent patent search found another American style football that is built with an electronic circuit coupled to an inflatable bladder. Interestingly, the football in this patent is designed intentionally with the electronics causing imbalance, unlike the above designs where the creators made sure their balls are balanced. Even though Wilson Sporting Goods has been granted this patent, there has yet to be any news of them releasing an instrumented oval ball. This might be something to look out for?

Ball Movement Measurements

No smart ball is complete if there are no “smarts” involved. The acceleration and/or angular velocity that is measured do not mean much if they are not processed and analysed. So firstly, the inertia sensors would require calibration – to ensure that the measurements are linear and accurate or at least corrected based on a benchmark device. Then mathematical models would be derived to determine the parameters for analysis; parameters such as spin rate, spin axis, speed, timing, ball flight path, angles, point of kick, bounces etc.

Also, to ensure that relevant data is processed accurately, certain “markers” or references are put in place to indicate when ball movement needs to be analysed and how it should be analysed. For the smart cricket and AFL ball developed by RMIT, as they are still in the research stage, a lot of the sensor measurements, signal processing, calculations and analysis are done manually. However for the commercial products like 94Fifty and the micoach smart ball, they have developed algorithms as well as guided user interface and instructions to make sure that each throw or bounce or kick is analysed accurately. In both cases, the interfaces and algorithms come in the form of an iPhone or iPad app. Here’s a breakdown of how each ball does it:

Basically to analyse a kick with the adidas micoach ball, the micoach app needs to be turned on and connected to the ball via bluetooth. Then after the ball is positioned stationary on the ground, the user has to select his/her kicking foot and tap on the ‘Kick it’ screen before executing the kick. One condition for getting the parameters measured is to kick the ball at least a metre off the ground and for it to travel at least 10m. No bouncing or rolling kicks.

Similarly the 94Fifty ball requires its app to be turned on and connected via bluetooth for the shots to be measured. For measuring shots, the user’s height needs to be entered into the app as well as the distance where the user is shooting from. There are options in the app to utilise a shooting machine or a user can practice with a training partner who can pass the ball after each shot. The only condition is that the pass has to be a chest pass for the subsequent shot to be recognised by the app. There are also some workouts or skill trainings that allow users to practice on their own and ball handling tracking options.

The Coaching Element

All these sensor-laden balls and their accompanying apps with smart algorithms aims to help users become better players – whether it is improved technique in kicking or shooting or training of muscle memory to perform proper mechanics over and over.

The 94Fifty app provides real-time audio feedback for each shot that a user makes, whether the focus is on shot arc angle or shot speed or shot backspin. Based on ideal stats (e.g. arc angle of 52 deg and backspin of 180rpm), the user can fine-tune his/her technique to achieve the right angle/speed/backspin. This user shows how by utilising the app’s feedback and capturing his practice on video at the same time, he could analyse his shot mechanics and identify how he could correct his shooting technique.

Likewise, the adidas micoach smart ball app not only measures each kick with ball speed, spin, spin angle, ball strike location & flight path, it also provides “Coach Notes” with recommendations on how the user can boost each specific parameter. A video option within the app allows a second person to capture the user’s kick using the iPhone/iPad’s camera so that the user not only gets the kick statistics but also visual playback of the kick.

Bottom Line

Designing a smart ball that analyses a player’s performance is definitely a complicated process. Not only must the instrumented ball behave like a normal standard ball with proper balance, but the electronics incorporated within the ball also have to be held robustly so that they don’t break under impact and the sensor data remains repeatable and reliable. Then there is the task of working out what parameters can be determined from the sensor data, if constraints/markers/references should be put in place to ensure accurate measurements, and how those parameters are helpful for improving an athlete’s skills and techniques.

Even with a properly designed ball that measures all the critical performance parameters accurately, it’s probably still not a complete coaching system. What the ball (and app) lacks is the ability to know (and break down) what exactly the athlete did in his kick or shot to achieve the numbers as calculated by the app. For example, in football, what affects a kick include foot speed, which part of the foot kicked the ball, and the amount of upper-body movement; and in basketball, a few things that influence a free throw include: the amount of trunk and knee flexion, shoulder flexion and elbow extension. These range of movements could be tracked with either video analysis (such as Kinovea which is markerless) or a 3D motion tracking system (such as Vicon which requires markers), or wearable sensors (such as SabelSense, XSens or this new sensor embedded compression suit).

In a nutshell, smart balls are definitely great coaching tools. But if combined with athlete movement tracking, it would give a lot more insight to improving the athlete’s shot performance.

Re-Engineering Labs

Smart ball