August 18, 2016

Virtual Reality

Virtual Reality in 2007 was different to what we know today.  After graduating university I landed a job at a company in Nottingham that called themselves True2Life Ltd, a sister company of Optima Design Services Ltd.  Catchy name.  They sold interactive virtual reality simulations that played over the web, CD-ROM or delivered wrapped in an app.  Prior to uni I had worked with 2D Flash quite extensively and used ActionScript to make things interactive.  It was an amazing technology at the time.  However in the early days it couldn’t do 3D, it was just too slow on the average business computer.

True2Life Ltd had partnered up with a little known company that produced a 3D ActiveX control called TurnTool.  You could export from 3D Studio Max to a turntool file which had a scripting interface to allow Javascript to send and receive messages between the web page and the TurnTool ActiveX Control.

The first job I worked on was a Conservatory / blinds configurator.  You could select your style of conservatory from a huge list of styles then the 3D view would take you inside a house where you could try out different blinds.  It was all in 3D and worked in your web browser with hundreds of configurations.  Back then the most difficult task was making the UI CSS work across browsers.  It would look great in IE but misaligned in FireFox so we spent a good deal of time creating custom CSS depending on which browser you loaded.

TurnTool  – 3D Web

The Life of Iron.

The project brief was to create a generic steel plant that could be used by members of International Iron and Steel Institute around the world. Members use the plant for a variety of uses from training and education to promotion of the steel industry.

The team consisted of several 3D artists and web developers from China.  Eventually they left the UK to head back home so I worked solely on versions 3, 4 and 5.
steelworks
International Iron and Steel Institute wanted a new building adding to their VR Iron works so I headed off to Talbot and had a grand tour of a cold rolling steel mill.  It was fascinating watching huge industrial machines with miles of flattened steel flowing through machines at break neck speed.  The steel was stretched, flattened, shaped and moulded.

With SLR camera in hand I took around 400 photo’s of the process.  Back at the office the photo’s were organised into categories so that each machine could be re-created in 3D Studio Max, including animation then placed where it should be in their 3D world. Not only was the Life of Iron a showcase of how steel is produced, it was a game too.  You could freely run about the large steel mills inside and outside in the open collecting items on the floor (hats, coils, shovels etc).  Once you had collected 5 hats, a door would unlock allowing you to move into another zone to learn the next stage of cold rolling.  It was a huge job broken into multiple levels so as not to overwhelm the users hardware.
We did not have asset streaming back then so things had to be managed carefully.  We even created a live overlay UI in flash that would talk or listen to Javascript messages. It was cutting edge.

Other jobs consisted of product configurators.   One to mention was a company called Altecnic that had a tap assembly.  They wanted to show in 3D how all the parts fitted together and in which order they went.  It was like a maintenance tool.  One great thing about TurnTool was you could isolate animation allowing a model part to animate keyframes without running the entire animation timeline.  Something tools like SketchFab cannot do yet.

Altecnic video capture running within a browser.


Charnwood, a company that produces beautiful stoves wanted a configurator to show each stove and the various options.  They also wanted a room that a user could play with to roughly match their own decor and see how a stove would look.

A note on CAD conversion:
The stove models were all raw CAD files so not suitable for realtime 3D.  A workflow of importing CAD data, which is a tricky job itself involving several tools to get the optimum imported model into max so that repair and poly reduction could be made.   CAD conversion almost always produces defects in models with flipped faces, vertices racing off into infinity, holes, double edges, broken verts, welding issues, smoothing issues to name a few so it is imperative that the original file has been exported with the correct settings (if possible) and then the toolset you use to go from Solidworks/Pro-e to something that 3DS Max can use has been fully worked out.
Max these days can import a variety of mathematical cad formats like Breps and Body objects.  In the past I have used Okino polytrans software to import models which was fantastic.  Especially when dealing with Pro-E assemblies and such.   You should always have other tools at hand though as some models just refuse to convert in Max or it brings in 99% and that 1% takes you ages to rebuild.  Its not worth it when you charge per part so have extra workflows you can switch too in an emergency.

Here is a video capture of the stove configurator running in a browser.

So, that was the days of TurnTool.

Unity3D – iOS Game

I started to look at the Unity web player to see if we could replace TurnTool.  This was a giant leap for 3D web features.  An IDE that imports straight from Max and allows you to reposition models, add lighting, animation, textures and with several different supported programming languages.  Manipulate the world as you like and it worked in a web browser!  I was in heaven.

Back in 2011 I wanted to test out Unity3D and see how an iPhone game was produced,  DropShape was born and published to the Apple App store.  Although this game could have been written in native Objective-C, it was a good test to see how Unity3D worked.  Unity was great although the download size at 25MB for a simple iPhone game was too big.  Native code would have got that down to around 6MB including all the voiceover assets.

dropshape

Now that I had worked out how to use Unity3D, I convinced Optima to buy a full copy for the next project.

Unity3D – Health & Safety Windows/Mac desktop app

With an understanding of Unity3D we chose it for an upcoming induction video for the Orion Laser Facility.  The induction video has a scripted health and safety talk to go with the 3D animation.  This was a 20 minute animated simulation which is a long time in the world of 3D animation.  Unity3D was fantastic to work in compared to TurnTool.  The simulation took place in a huge building with over 60 scenes, character animations, professional voice over narrative music and hundreds of models.  The storyboard took a few months to refine with several iterations and ended up about 200 pages long.  The project began, all code was written in C#, all models, characters and animation produced in 3D Studio Max. Finding the right voice for narrative can be quite tricky. The natural tempo, clarity, seriousness and above all an engaging voice to keep viewers attention had to be found, also, using a professional studio for recording produced a perfectly balanced audio file with no noise, muffle or other artifacts.  The project lasted just short of a year.  7 years later and I am now on version 5 of this project with new content and updated H&S information.

Screen Shot 2016-08-23 at 12.48.13   Screen Shot 2016-08-23 at 12.48.55   Screen Shot 2016-08-23 at 12.51.53

Unity3D  – Nite Web3D – MintViz

I got a call from the studio MintViz asking about interactive 3D and could I take their very high poly models, match the materials and lighting and allow a user to interact with it via the web.  Now the guys at MintViz have an eye for accuracy when it comes to 3D renderings so quality is their highest priority.  Using Unity3D and an Image Based Lighting (IBL) plugin called Marmoset Skyshop I was able to get a close match to the Vray rendered beauty shots.  They also wanted the watch to tell the correct time and auto update,  show an exploded view and show how it looks with the watches unique day/night mode of lighting.  I used C# for the interactivity and made them a quick prototype.

Unity3D’s web player is an ActiveX control which is now being depreciated across all browsers (FireFox still allows it, just) so a web standards implementation of Web3D needs to be found.  Web 3D is almost there with tools like SketchFab and PlayCanvas but they lack features found in Unity at the moment.  I am watching Unity’s own implementation of Web3D closely.  Unreal Engine also allows Web3D but it takes far too long to initialize on an end users web browser.

Oculus Rift / Vive

We now have VR headsets that are a complete game changer.  No longer do you sit outside the screen but now you can sit or stand inside the simulation.  Woaaaaaa.

Oculus and HTC have pushed VR to a whole new level.  I bought a development Rift headset (DK2) to see what all the fuss was about and it blew my mind, made me vomit, gave me the shakes and I still came back for more.

At the time Unreal Engine had better support for the Rift than Unity so I started to learn how UE4 works with their BluePrint system.  It was kinda neat, a bit like Playmaker for Unity3D which is known as a state machine.  You connect function nodes together visually instead of writing code.  It got somewhat confusing when node trees got huge but it was a fun way to work and could get really powerful.  The alternative was C++ in UE4 which I gave a wide berth although I may go back and give it a go.

imgres imgres-2

So In 3 weeks after picking up UE4 I created a hazardous 3D lab.  My intention was to show how simulations can now create real fear for the viewer when used with a headset.

The lab had a 20 foot floating animated logo at the start with football sized steel balls spinning around it, the balls had sound effects attached to them so would buzz past your head as you walked under it. Instinctively you would duck.   it included an emergency eye wash shower that triggered when you stepped on a plate, try looking up at the water raining down on you without blinking, it was almost impossible as your reflexes fought against you.  Then there was a laser beam that if you happen to walk in front of it your character would scream and smoke would rise up from the point of contact, fire hazards with burning flames raging across the floor and the loud sound of a fire out of control, chemical spills and other scary stuff, when I tested it out on a client they were initially amazed but I made a crucial mistake.  The last room was dimly lit and had a huge pulsating particle system right in front of you, you had to walk through it.  With the headset on this felt like it was 30 feet tall so quite imposing on the viewer however the real problem was the lighting.  The viewer, after staring up at this scary looking pulsating particle cloud got disoriented as they couldn’t quite see the floor or walls to “ground” them.  The client took the headset off and went “urrrr, I feel sick”.  so that was the end of that.  However, with the advancement of the latest headsets from both Oculus Rift and HTC Vive and the known no-nos of what to avoid in VR I think I should probably push this again for serious games / simulations.  It is immensely fun work and with such a powerful lasting memory imprint, it is the perfect medium for simulated training.

The above video shows some of the Oculus Rift content but you do not get the same immersive feeling watching a video as you would if you had a headset on.  With a headset it really does feel like you are there, inside the video and that water will hit your eyes if you look at it (as this is a special eye shower, you need to keep your eyes open!) and the belief that fire will burn you if you move near it.  For health and safety training this is very powerful.  Add in some live events where the user has to react to changing conditions and have the computer grade them on their choices and response times.  The training will also stick in their mind longer due to the realism.