Fuelr - real-time video training sessions with a real trainer

digitalhealth

#1

Original Post: http://www.steveschofield.tech/techbiz/2014/3/27/reigniting-an-old-flame-exercise-service-delivery

When I was 18 (I think, it was such a frickin’ long time ago), I used to sit in office buildings for 12 hours straight, working site security to pay for my university fees (well, beer money). I was fascinated by the idea of offering prescribed physical activity for people with chronic conditions, and for the prevention of disease in at-risk populations.

I had completed a module in my sports studies degree, on this very subject. The NHS had not long released the framework for quality assurance, for the prescription of exercise and exercise referral schemes.

I used to write business plan after business plan, how I would construct a company to offer such services. I would talk to GPs and practise managers. I would do cashflow forecasts and P&Ls. I would research the equipment needs, and create startup cost budgets.

Then I did nothing about it. Until now. 14 years later.

Not only am I now working on a next-generation Social Care platform, which is all about re-enablement and self-management of conditions, but I’m also working on a health and wellness platform for tracking and improving your overall wellbeing, by improving your knowledge and behaviour through the use of smart technologies and connected devices. I am now positioned to build something that does what I wanted, only a decade ago.

But, this is not the same plan - such has been the journey and has lead me back to this point. My education, in particular over the past 4 - 5 years, has armed me with what I was missing back then.

  • Technical skills and the ability to execute.

  • The mental processes that force me to ship a product

  • Confirmation that my passion for this space is unyielding.

  • Knowledge of a global market opportunity, enabled through technology, for distribution and scaleable business models.

The original idea was nice. But it wasn’t scaleable. It was a lifestyle business idea, like a hairdressers, or an activity centre owner, or a restaurant owner.

I was in San Francisco 2 years ago, when I saw the idea, realised, working and gaining traction. A US startup called Wello had pulled together nascent real time video streaming protocols, such as webRTC, and modern web application development techniques, to offer a marketplace for exercise professionals.

http://static1.squarespace.com/static/5005c0c6c4aa8b4d9761ee53/t/533bb374e4b0a80ee4d0ee91/1396421495007/wello.png?format=1000w

Go to the site, and you can take part in pilates, yoga, circuits, cross-training, strength training… you name it. Delivered by real experts, in real time, at any time, wherever you or they are.

Now, this really appeals to me, both conceptually and technically.

So, this thread of posts is going to follow my exploration into building a very similar service offering, with I think, some key differentiators. We’ll tackle the main technical concepts, including:

  • Building a web services / API using, most likely, Ruby on Rails
  • Building a native iOS application, specifically for iPad
  • Using webRTC and the different options available to make this easier
  • Creating an admin area for the business
  • Creating the web application front end views

#2

“The goal of a #leanstartup is to move through the build-measure-learn feedback loop as quickly as possible.”
— Eric Ries, Lean Startup

And so, with that in mind, I’ve been pushing to get Fuelr into the hands of real users as quickly as possible.

If you’re not familiar with the Fuelr project, you may want to jump back :arrow_double_up: for an overview.

The Fuelr iOS app, for iPad, although far from production ready, is functional - or so I hoped. Actually, until the live session, I hadn’t even seen some of the key features of the group video sessions working in development so I was a bit twitchy.

##Goals

The goals for this live trial were:

  • See if the current prototype was sufficiently functional to deliver a real training session.
  • Test the product with dubious broadband connectivity
  • Test an initial content idea for a training session
  • Test if the end users actually liked this kind of workout format and delivery
  • Figure out some guidance / rules for the optimal physical setup for the trainer and the user
  • Test the fidelity of the video stream
  • Test the suitability of the audio

The Session

Setup with Testflight

The first issue with any such pre-release test is not so much the app itself, but getting normal people setup with the app, via Testflight, so that they can install and run it. It always seems to create some confusion, but generally for this test, it was fine. The attendees made light work of the Testflight setup. That is, except one poor tester who suffered at the hands of some hidden iPad preferences, which wouldn’t allow Testflight to install. After 15 minutes of debugging by phone, we had to abort and continue with the test session, which was a shame.

For those struggling with this issue, this article provided some tips to resolve, but alas, they didn’t work for us.

Physical Setup

I always had in my mind that the physical setup was going to be key for the smooth and seamless delivery of the training, the communication, the relationship between trainer and client. Factors I was considering, included:

  • Relative height of device to the trainer
  • Relative angle of device to the trainer
  • Distance of trainer to the device
  • Effect of above on audio fidelity and the potential need for external microphone
  • Lighting and effect on visual fidelity
  • Optimal contrast between environment background and the trainer/clothing
  • Amount of general space required for trainer to demonstrate exercises effectively
  • In reality, the iPad is bloody fiddly to position in any way that is not the specific design of whatever general purpose case you are using - there’s normally one or two position variants and anything else will just mean that it falls over.

At this point, I’m less concerned with finding the correct case / stand solution, but more concerned with figuring out where the ipad needs to be in relation to the trainer for optimal quality of the session.

So, with the help of a step ladder, a blanket, some cushions, a magazine and and elastic bungee cord - we had our makeshift way of getting the ipad at the correct height and angle to correctly frame Jo, our trainer, in the video. We could also quickly adjust the angle, great for the shift from standing to floor exercises, by inserting a magazine behind the ipad which was under pressure from the bungee cord and so stayed nicely in place.

We found that it was better to have the device lower down, around 1 - 2ft from the floor, angled up slightly for standing shots. Though, I neglected to measure our final setup, note to self, do that next time.

Needless to say, not a scaleable or practical solution, but very useful for the purposes of determining the guidelines and possibly creating a low cost solution that could be sent to trainers and users to improve the overall experience.

Broadband Connectivity

From the trainer side, we ran the session from a fairly poor 6mbps down / 0.5 mbps up ADSL line. At Jo’s end, we had a couple of issues with the video stream freezing, particularly when we had more then one device connected to the session on the same network. When we killed the additional device connections and refreshed the session, we got through the entire 2nd half of the session without issue.

For the clients on the other side, of whom, I know at least one had a good 60mpbs down / 18mbps up connection, they reported no issues of freezing or lost connection to the video stream. “It’s just like Facetime…” said one of the participants, not sure whether that was compliment, at least she didn’t say Skype :wink:

Content

My challenge to Jo, who is in charge of the operational side of things, was to think about the style and tone of the content. To think about what type of sessions would be compelling, engaging and fun when delivered. Our initial session was a highly targeted, dynamic 30 minute glute workout, shaping those buns for the summer ahead. After a quick warm up, there was a smooth transition into the main workout and then into the cool down.

The content was good, had zero dependencies in terms of equipment and the guys had a great time during the session.

Social Experience

A really interesting side effect was that, through the Fuelr session, the people training who didn’t know each other before hand, really engaged with each other and with the trainer in a way that was unexpected. This was a really positive outcome and one I’m keen to investigate the dynamic of in future sessions.

To Sum Up

Really pleased we simply managed to successfully complete a full session, via a 3-way connected iPad network, with people who didn’t know each other - engaging around a common goal - to tighten up their buns!

Can’t wait to push this further and see where it leads.


#3

Fuelr API V1

I initially created a basic rails-based API for the Fuelr app.

It had all the basic resources needed:

  • Users, auth, registration & profiles for customers and trainers
  • Products - the types of activities on offer
  • Slots - the times that an appointment can be created for a product
  • Appointments - an instance of a product slot that brings together a trainer and one or more customers.

I was having some problems with the OpenTok framework and in the end, the rails API completion took a back seat whilst I figured that out.

I created some views for doing some of the administration of the app from a Fuelr team member perspective, with basic CRUD capabilities.

I even started to write some documentation - yes, it got that serious!

Version 1 of the API is on hold until I can nail down which WebRTC platform to use, because OpenTok is not playing all that nicely.


#4

Fuelr’s Simple UI

The initial concept for Fuelr was always an iPad app. I felt the iPad gave us enough screen real estate during the exercise session to support multiple windows streaming video content from members of the class, including a window for the instructor and the current user themselves.

I really wanted the app initially to feel quite fresh, personal and perceptively simple to use. I purposefully wanted to keep out a lot of things, in favour of simple navigation and a clear reason to visit the app each time - normally to select and then take part in a session of the user’s choosing.

The flow would be very straightforward. After a nicely animated greeting view…

Users don’t even need to register, to start browsing the available sessions.

When they find something they like, such as a nice group Yoga class, they are prompted to register for an account or login.

The user will create a simple profile…

… and optionally, connect any and all of their health & fitness trackers / devices / sensors at this point.

An account will unlock credits that can be used for appointments, the number of credits and types of appointments may depend on which account plan the user has purchased.

The app will notify the User with a push notification in the lead up the the appointment time, and 5 - 10 minutes before the appointment’s scheduled start time, the session Entry button will be activated and accessible, so the User can enter the room and get setup in advance.

As you can see from this last screenshot, we’re leveraging the Nuwe NUSCORE API to graphically display the User’s current health score, which the trainer will also be able to see and have a very quick visual understanding of the User’s current status across physical activity, nutrition and biometrical performance versus the User’s personal goals.

If the User has granted deeper access to the data, we will also provide some visualisation tools for the trainers, but that’s out of scope for now.


#5

I did a write up last year about Accessible 3D motion technology, using the standard 2D webcam embedded in mobile devices to do motion tracking for exercise applications.

It made sense to think about applying this technology within the Fuelr concept.

So, let’s remind ourselves what this is about.

We’re using the iOS library from ExtremeReality to real-time interpret the movements of the skeleton, using our iPad camera.

From the Startup Bootcamp office in Barcelona, here’s an example of it in action, with a very quick boilerplate project I created that turned head movements from zone to zone into a score. Thus resulting in Blai Carendell’s moshing.

So, conceptually, how can we apply this in Fuelr?

My immediate thought is that real-time motion tracking can be used to provide real-time feedback as well as contribute to post-hoc analysis after a session.

It’s often difficult for instructors, who may be managing sessions of 6 - 30 people at a time, to really hone in on the details of how to improve movements, techniques and general performance after the fact.

This is really important, not just for the continuous improvement cycle that customers expect, but also to prevent injuries from repeatedly mis-performing moves.

So, we can use this technology perhaps to augment the teaching and feedback, providing reporting tools on a per user basis for the instructor to use when concluding a session, or to raise alerts to the instructor requiring intervention in real-time.

This technology could also be used to add a fun layer of “gamification” (for want of a better word) whereby accurate movements, correct body angles, the number of moves or sequences performed could all contribute to a kind of performance score.

What would it look like?

Well, since you asked, maybe something like this…

Let’s take a fairly normal Yoga session.

And augment it with 3d Motion tracking…

Then, in real-time, start to count the moves, and score based on accuracy of body position throughout the session.

So that when the session ends, the scores are totalled and the session leaderboard is shown. In addition to this, we can provide automated insights, based on the session data from the 3D motion tracking and also from any fitness trackers the user is wearing, along with, of course, anything that the instructor wants to add themselves.

What do you think? Would this work… would it be fun or useful?