AI in animation

This last week I’ve been at the Manchester Animation Festival - a week-long festival for industry, education, and public alike. The panels and short films lit my mind up with an inspiration and drive that I haven’t felt in a while.

On the Industry Day, one of the panels I’d most anticipated was about AI: the fears, the predictions, the ways we’re already using it to improve our work… I’m certainly using AI tools in my photo editing, and as a base for subtitling any fast turnaround work for YouTube etc

For my money, a lot of the practical uses will end up being based on personal/client preferences, past the obvious ethical and legal conversations that are already taking place.

Take voice acting. When we change a line / scene in a script, we usually scratch in the dialogue by getting someone in the studio to record it. The actor then records the line at their next session, or ADRs it if it has already been animated to the scratch.

The current ethical debate, prompted by existing “machine learning” in which AI systems have been “trained” on an actor’s voice and then prompted to generate new speech from a written prompt, is “we can replace that by training an AI on the actor’s voice” vs “actors should be paid, and given the chance to produce the most authentic work”.

Obviously I agree with the latter- audiences deserve the authenticity, and suggesting that AI can replace their training and talent is horrible. But. How about using the AI as a temporary placeholder, instead of the scratch of a completely different voice just jumping in?

+ you get a smoother viewing experience

- the output is misrepresenting what you actually have “in the can” (burn-ins are easily overlooked)

+ it joins up more neatly, and you can judge timing better with a more accurate placeholder

+ if animation for the shot must happen before the actor is next available, a smooth read in the correct cadence and pronunciation will help lipsync

- the actor may be unnerved by hearing their own voice at ADR/ sense the uncanny valley more readily

… all of this assumes that the technology allows for cadence, tone etc. But existing technology such as VocAlign/ Revoice Pro (where you can match an audio clip by the actor to a reading by anyone else) take us a long way towards that already. We can use a written “find” command to find words or syllables in an audio library in Avid’s PhraseFind or DaVinci Resolve Studio. I feel confident that I could get some way towards a useable voice performance… if given enough time. If AI can be used as a tool to cut out that time, and present options to a human, it could be very useful indeed. IF it is desired.

It’s going to be an interesting time. But whether people currently view it as the dawn of a new era or the opening of a new Pandora’s Box… AI is happening. It’s worth being part of the conversation.

GBFTE First Frame - "A View to a Skill"

This was a piece written for the magazine distributed by the Guild of British Film and Television Editors, with my former trainee Mary.



A VIEW TO A SKILL

A Year on the Skillset Trainee Finder Scheme

by Judith Allen

 

Trainee Finder is a service that matches trainees with companies across the UK’s animation, film, games, highend and children’s TV and VFX industries. The scheme is supported by Creative Skillset’s Skills Investment Fund which encourages co-investment in skills and training to ensure a continued supply of a new generation of talent, capable of world-class creative content.

THE EDITOR : Judith Allen After graduating from The National Film and Television School, I worked at Aardman for a few years as the previs and VFX editor on The Pirates! In an Adventure with Scientists. Working on the previs allowed me close contact with director Pete Lord and cinematographer Frank Passingham, both of which were invaluable in my future editing. After Aardman I got my first series as editor, at Blue-Zoo cutting Q Pootle 5 - which led onto working on Miffy and Digby Dragon.

THE TRAINEE : Mary McKeogh I have always been very keen on stories and storytelling, so it was very exciting to discover I could go to Ravensbourne University and work towards a degree in editing. Three years later I graduated with first class honours and was fortunate to already be employed running at Envy Post Production. Working as a runner was great experience and gave me my first insight into the industry outside of a lecture hall. My next two positions were with the Skillset Trainee Scheme first working at Red Planet Productions as a post production assistant and then at Blue Zoo Animation where I received my first editorial credit on Miffy’s Adventures Big and Small and have been working there ever since.  

 

How did your involvement in the Skillset Trainee scheme come about?

Mary McKeogh (MM): A few short months after graduating, a family friend working as a post-production supervisor for Red Planet offered me a role as her assistant through the Skillset Trainee Scheme. I took the job seeing it as an opportunity to experience something different to running, but also as a chance to gain access to working edit rooms where I could meet editors and learn from their assistants. Being a part of the Skillset Trainee Scheme meant that my employer ensured that I did get to spend time in the edit rooms to watch, work and learn from the team there. I was very keen after this experience to land a job as a second assistant/junior and work my way up. One of the many, many emails I sent out during this job-search reached Judith’s inbox shortly before Christmas 2014. She replied (a reply!!) letting me know about an opening for a year-long placement as an Skillset editing trainee at her current place of work - Blue Zoo Animation.

Judith Allen (JA): Mary’s email came at the perfect time. We’d decided to look for a trainee to help with the workload of two series at once, and I’d recently made a series of blog posts on my personal website that I could link her to and show her what the job would actually involve. Blue-Zoo had taken on Skillset trainees in other departments before, so they knew what would be involved on the administrative side of things, as well as the quality of candidate they could expect to get. We interviewed three people in the end, and Mary stood out to me for her previous professional experience - especially having gained so much during university, and for her commitment to editing and story-telling generally, over the focus on animation that some of the other applicants displayed. It’s often the case that people seem to know a lot less about animation editing than other forms.

 

Why did you choose this specialism?

JA: My introduction to animation editing was at the National Film and Television School, where I was on the MA Editing course. We worked with the animators a few times in the first year, and then on their graduation films throughout the second year. I enjoy the creativity that animation allows, and the involvement of the editor through all stages of production - not just coming in at the end when almost everyone else has finished their job. It allows for shots to be created and tweaked as we go along, and the editor gets to collaborate with all the departments.

MM: I have to admit until Judith’s reply popped up in my inbox I had never considered animation as a possible career path. At University they told us about many possible jobs we could do under the post production ‘umbrella’ but animation never came up (I might have been in bed that day). It came as a complete surprise to me when Judith offered me a place to train under her at Blue Zoo. Whilst my previous experience with the Skillset Trainee Scheme had been so positive, I did have my reservations about becoming a trainee again. I was eager to move up the career ladder and wanted more responsibility than my previous roles. On the other hand, I was very intrigued as animation is such a beloved, interesting art-form and in the end I just couldn’t turn the opportunity down.

How were the first few weeks?

MM: At my interview I found Judith to be very reassuring. She had worked in live-action and features before and offered to share her experience with me whilst training me in animation editing. I was offered the role. I accepted and I started working at Blue Zoo in January 2015. I wish I could recall the first few weeks there but they honestly went by so fast! Working in a studio was different to anything I had done before. I got to sit at my own computer (my own edit desk!) and was quickly given lots to get on with. There were so many people in the studio to meet and work alongside - learning who the directors were, who the other directors were and why were there so many directors?!

JA: I had trained an assistant before, on Pirates, but that was a very different job with a huge focus on some of the more technical elements of how to help with what we were doing, and where we fit in on the bigger picture. We still needed to address that here - but we had already set an end goal of having an editor capable of taking on a series creatively as well as technically, and being a good investment for the company. So that meant taking the time to get a solid grounding in the animation industry, with an introduction to all roles and how they related to the things which came in and out of the editing room.

MM: I am completely in awe of what goes into making an animated production. We’ve all sat through the long credit scrolls listing endless names and departments but to witness it in action is truly fascinating. As a training animation editor I was encouraged to get involved with all areas of production. I got to sit in on storyboard meetings watching the artists sketch and map out scenes and characters, observed modellers experiment with character design and movement, saw animators bring these characters to life on screen and listened to incredibly complex conversations amongst lighters and compositors as they discussed how to make these shots look beautiful. I was also given the chance to leave the studio and see how we collaborated with external companies on our shows. I sat in on voice records and mix reviews at the sound studios, attended online edits at post houses and got to attend various animation events to learn more about the industry and the people in it. I am still learning to this day and enjoy the complexity of each and every department that is involved.

 

How did you fill the time on your year-long placement? Did you feel a sense of progress?

MM: In terms of my daily responsibilities at Blue Zoo, it wasn’t long after I started that I began cutting together assembly edits of animatics for episodes. I would then pass these on to Judith to do her own editorial pass on before they went to the directors. Other tasks included creating ‘walla’ libraries from the voice record sessions and updating episodes with the latest shots from animation and lighting/composition. As time went on, I found myself taking on more and more. I would edit animatics and then work directly with the directors to tweak and trim the episodes. I would take feedback from the clients and look after episodes through the delivery stages. I felt that every month that went by I was trusted more and more by my producers and directors and my confidence grew alongside that trust. By the end of my trainee year I had advanced my editing skills beyond what I imagined and had fallen in love with animation. I was offered a contract at the end of my Skillset Trainee Scheme and have been working for Blue Zoo ever since.

JA: It had always been our goal at Blue-Zoo to train an editor, rather than an editing assistant. Our main series through the year was Miffy’s Adventures: Big and Small, based on the Dick Bruna books, which had the same air of minimalism and simplicity - whilst needing to be expanded out into six-minute stories which could hold a pre-schooler’s attention. In many ways it was the perfect show to bring a trainee in on, and laid many of the creative and technical foundations for when we moved on to Digby Dragon - a 52 x 11 minute series with greater complexities in most aspects. I’ve tried to keep giving Mary creative work on this series, but she’s also been gaining a lot of experience of what an assistant editor on a larger animated production would do. Looking back, what did you enjoy most about your trainee year?

MM: It’s great to better understand the animation process and where my role fits in, getting to sit in with different people in the studio and listen to them explain their work was really interesting. It was fun to escape from the edit room and hang out at the sound studios watching the voice records. Now I’m not a trainee anymore I have no time for escaping! I’d also like to say that overall my favourite aspect of this role is the teamwork, a team in which you know just how hard every single department works and the respect that comes with it. I find in live action that once filming is over, everyone moves on and the post-production team are left slogging it out until the end so the editing room can be a lonely place. In animation you end up working closely with storyboard artists, producers, animators, compositors, directors and sound mixers the whole way through. Judith’s and my workstations are the only place where the episode lives as a whole. It’s where everyone’s work comes together - so you get a lot of visitors over the day! 

 

What’s next?

JA: I’m currently arranging for Mary to get some experience and contacts amongst people I know who work in pre-vis and VFX - both of which are related closely to elements of animation, and bridge that gap into live action production. I’ve been lucky enough to work with both in the past, and I think they’re all intricately tied together in very cool ways. The advantage of working in London is that a lot of this sector of our industry also works here, and eventually you end up bumping into the same people again even if you’ve all switched jobs several times on the way!

MM: Currently I am contracted at Blue Zoo until autumn 2016, whilst assisting Judith on our bigger production (which we have recently finished delivering the first half of episodes for). I have also been given the responsibility of editing a shorter web series for WGBH in the US. It’s a great production for my first solo-editing venture and have had to navigate my way through the complications of working with NTSC 29.97fps. I’m hoping to stay with Blue Zoo for a while more, I would like to step up to editing a bigger production and am using my time there to further educate myself on Avid Media Composer (I’m mostly an Adobe user). In the long-term I can’t say I’ve closed the door on live-action. It’s also a strong aspiration of mine to work on a feature to experience the feature film environment and find out if it is something I want to pursue. I just love stories and I want to work on as many stories as possible, long ones, short ones, fiction and nonfiction. I would love the opportunity to try them all. I’ve recently signed up to Sara Putt Associates Trainee Scheme. I meet up once a month with 30 other trainees from a wide range of departments including camera, costume and sound. We go through the various skills a freelancer needs in the television and film industry and I see it as a great way to meet others at the same stage in their career as me. The networking opportunities are great and I leave every session feeling freshly motivated to work hard and aim high.

 

Miffy’s Adventures: Big and Small is currently broadcasting on Tiny Pop in the UK. Digby Dragon is due to start broadcasting on Nick Jr. later this year.

Judith’s website - www.jaa-editing.com

Mary’s LinkedIn - http://uk.linkedin.com/in/mary-mckeogh-19267a88

Progression of a timeline: animatic

For several years now, the Twitter editing community have been doing #TimelineTuesday - in which you screengrab the timeline of the project you're currently working on, and share it with the other editors. To us, it's an interesting look at the way other people set up their timelines, and some pretty big editors have joined in in the past, sharing various reels of major feature films with varying degrees of VFX and sound work.

I appear to be one of the few people sharing animation timelines, and they're quite different to what most other people are used to seeing. Indeed, they're quite different to my own live-action timelines (a comparison briefly discussed as part of Avid's own #TimelineTuesday series).

So, I'm going to show how I get there. What the various stages are, and a brief explanation where possible of what's changed and why. A previous blog post has a flow chart which shows the 'typical' route through the edit of an episode, for reference to how each part sits in the whole.

I'm using an episode from the series I'm currently editing as an example, and this post covers the animatic stage of production (for additional information see "editing an animatic", a previous blog post based on a different series). For animatics, I use Adobe's Premiere Pro.
Read more

Avid Blogs - "Editor Judith Allen Illustrates the Difference Between Cutting Animation and Live Action Films"

Excerpt:

In live action filming, every action on set is accompanied by the corresponding sound. Often these are replaced or enhanced, but the sound is there and used within the rhythm of the cut whether consciously or unconsciously. In animation, every single sound needs to be purposefully placed – and often created from scratch by a foley team. It’s often a delicate balance between knowing which temp sound effects are best to place during the animatic to help tell the story and do the best cut, and which ones should be saved until after animation so as not to restrict the animators.

At the moment I tend to make my sound choices based on how the characters need to respond to them in their headspaces more than how the final sound should be, because I know that the sound’s going to be completely replaced once we have the final picture, and because it will help the animation before that.

Read more

An Animation Editorial Workflow

I put this together pretty quickly for the internal project wiki at work as a basic overview of what goes in and out of my remit as editor on a 52 x 11 min animated series, where within the process sequence reviews typically happen, and where things come from/ go to.

It's not as entirely linear as this in practice (locks happen before all of the animation is done, for example - so that any extensions can be accommodated), a lot of things end up happening as simultaneously, and I'm typically working on anywhere between 5-26 episodes at a time... but this is the basic system.

I'm sharing it here because I get asked a lot where the editing happens in animation ("surely the work's all done by the time you arrive on the scene?"). Answer: everywhere. I haven't even included the script changes and pickups/ ADR here. They basically get added throughout, ideally less frequently as time goes on - the further we are in an episode; the more people are affected if something needs to be changed.

Read more

Cinema 4DX

4DX cinema is programmed by a South Korean company, building on the older theme-park style immersive experience (I have memories of an Alien 4D experience at a Disney park in which you could feel the simulated breath on the back of your neck as the speakers in the chair implied it was right behind you), but expanded out to mainstream feature films. The chairs move, and various effects of light, water, fragrance, and air surround the audience in complement to the on-screen action. Having recently moved out of London to the suburbs (greener, MUCH more affordable, ample opportunity to listen to podcasts on the rail commute), I find myself now living near the UK's first 4DX cinema in Milton Keynes.
Read more

The Making of Q Pootle 5

Q Pootle 5 was the animated television series I worked on over 2012-13. This 'making of' video was released on one of the series DVDs, and gives a basic overview of the process of creating an animated series: from concept art to foley and compositing.

 



For additional information on the editing side of things running through animatic, animation, and delivery, I wrote a series of blog posts while working on this series, which are linked to from a roundup post which includes examples of how the edit may change throughout production.

Article: "A Quick Look at Adobe Premiere Pro, and the Creative Cloud" - First Frame, Spring 2014

I was asked to write something for "First Frame", the magazine of the Guild of British Film and Television Editors about Adobe Premiere; who are one of the Guild's sponsors. The article was aimed at editors who were not already users of the software, and it was published in the Spring 2014 issue.
N.B. Pricing information was correct at time of publication, check the Adobe website for the latest pricing.
Read more