That's especially true of the show Schmidt put together for HD, his 2013 album as Atom™ on Raster-Noton. To the beat of wildly infectious techno-pop, a screen above the stage is blasted by abstract shapes, video collages and a heavily retouched approximation of Schmidt's head, which occasionally lip-syncs along with the tunes. The set is both a time warp to MTV's '80s heyday and a hyper-contemporary creation, which not too long ago would have required a small army of animators and effects experts to pull off. When we caught up at this year's MUTEK festival in Montreal, I found out that a range of disparate technological advancements, from ever-increasing processing power in laptops to the rise of YouTube tutorials, helped Schmidt, who has no formal training in video production, assemble HD in-house. After a short performance during MUTEK's day programming at the Phi Centre, Schmidt gave us the lowdown on what's become his signature set.
There's a computer just for the video and a computer just for the audio, basically for running larger parts of the audio. A small controller for the audio and I normally order a desk and an EQ for the stage.
Is there any audio coming from the MPC, or is it just triggering?
It's both, actually. Everything's done with the idea of reproducing pieces off the album, so the complexity was then: how would I fit everything into an MPC? And I think it's impossible to do that, or it would have been insane to attempt that. So I decided upon reproducing parts of the album as multitrack recordings from within Ableton and other more improvised parts, which just happened towards the end of that track, then run out of the MPC. So I'm basically switching back and forth between the two sources of audio. And, for example, the second piece I'm going to play, later on, is entirely coming out of the MPC—there's no [computer] audio involved.
How close is this set up to what you used when you were making the album?
Nothing to do with it [laughs].
So you completely adapted all of that for this. What did the album setup look like?
It's very much Pro Tools-based, it's all editing work. ProTools is basically the place where I end up putting stuff together, arranging music, mixing music. So where the audio comes from is—well, I'm flexible on that. So I have gear—synthesisers, drum machines, whatever—and I end up layering stuff in ProTools. So it's a very, very different process from what's here. It's much less improv-oriented; I play very little, actually. It's on-and-off, stop-and-go, letting things simmer a little bit and then going back to the recording, put it back together, delete stuff, and afterwards coming back. So it's a rather slow and not very real-time-oriented process.
HD was very, very slow coming together—something you've been working on for many years, right?
Actually, I had started on the album maybe ten years ago and then had a couple of technical problems and lost the mixes. I kind of started to mix it down and lost the mixes twice and then I got totally fed up with it. I couldn't listen to it anymore. But I had the unfinished mixes on my hard disk for like, ten years. I always dragged it from one hard disk to the next, and every couple of months, maybe, I would listen to some of it. And then, throughout the years, I actually deleted some of it. I had no connection to [the original] idea, and I just threw it away—there were two or three pieces off that recording that still resonated after, maybe, seven or eight years. But I had changed a lot, and my ideas had changed a lot, so I decided to adapt them and put them into something else, which then became HD. So it's not really that it's an album that took ten years or something. It's more like, I had tracks laying around and then adapted it into another idea, which was HD. I recorded the other seven tracks for the album afterwards.
Let's talk about the visual concept, which is the really striking thing about this set. How did the A/V set get started?
Well that whole A/V project started eight years ago. It came along when I realized that I was able to program and play audio and video all by myself, onstage. That was not possible to do during the years before, when I started making music and started performing 20 years ago. I always wanted to do that, but it was totally impossible—it would require a small team of people. I'm not very good at giving orders to people. Especially with video, I'm very picky about timing and stuff like that. And whenever I had tried to work with people, it was always—it didn't feel right. I always had to tell them what to do, and I don't like that. And then at some point, after I had performed for more than ten years with Señor Coconut—it was a totally different thing, me onstage with ten people—at some stage I was tired of doing that, of being onstage, travelling with people, being with people [laughs]. I really had this strong feeling I wanted to be alone onstage and be responsible for myself, kind of like a samurai idea. So I was getting really interested again in playing that type of set and did a little research and found out it was possible, actually, to do video and audio with a rather portable set, just by myself.
What had suddenly made that possible? Was it particular technology, a particular piece of software?
Well, laptops, I think—laptops and the fact that the average laptop is actually able to play HD video. I started to play with my MPC3000, which basically, all the audio came off the MPC3000 and the audio was triggering the video in real time. That was how everything started. So this set is a progression of that first idea. Due to the fact that I wanted to reproduce tracks off the album (which was not the case with the first set) I had to include Ableton for sound and be able to manage bigger amounts of audio data, and of course then I switched to HD video, which requires more memory and faster computers. So now I'm using two machines—two computers—to do that.
It's still necessary to have two laptops, one for audio and one for video?
You know, I don't trust computers [laughs]. It feels shaky in general, playing with computers. I don't like computers onstage. That's why, as you see, I don't really look at them. They could be somewhere else. They're not, because I don't trust them.
You like them within arm's reach, basically.
Yes. If something crashes, which has only happened, like, once in three years, I need to be there, to reboot or whatever. So that's why they're onstage. But for the first A/V, where I played with an MPC3000, I had the video computer somewhere else backstage. I ran it off a FireWire cable, and yeah, that worked pretty well.
So you designed all the visuals that we're seeing here?
Do you have a background in this kind of thing? How did you get started?
That was quite a regular experience, actually, because I was pretty sure about what I wanted to do audio-wise and how to solve that. And I had a very specific idea about the kind of visuals I wanted. So the option is always, "Do I find that person and explain to them what I want, or do I try to do it myself?" Actually, when I started to prepare that set, it took me maybe four months to actually finish the album. And to finish that production took me six months. That's because I had to learn video editing, blue-screening, everything from scratch. I bought the computers for the set. I bought the whole audio equipment just for that set and had to figure it out and learn it, and also learn the whole video side of it. Not only how to play it with the program I'm using, but also to make the video, to produce the video. So the first section, for example, of that video, the first seven minutes, are an entirely pre-rendered section that I trigger off the MPC and the two computers. So the audio comes off the Ableton, and the video comes from the other computer.
And they run in sync, fortunately [laughs]. When I first started to produce it, I wasn't sure if that was the case. Sometimes there's some glitch in the clock and sync or something, and it runs it out of sync. That's why the MPC is so important, because the MPC has a very stable clock and Ableton does not—sorry to say that [laughs]. If Ableton was the master clock for that set up, the video would not run in sync. So I needed a stable master clock, and the MPC is actually that. So I started playing the MPC, and there's basically two MIDI channels off the MPC. One is triggered with the video and one with the audio, and they run in sync forever, basically.
So MIDI keeps both the video and the audio in sync—that's the way that they're talking to each other?
Yes. So the MPC is starting the clips and switching the clips—switching the video programs, switching the scenes, the layers and so on. And every now and then I can play the clips with pads, and sometimes they're playing by themselves from within the MPC.
So you're still getting to play the video? This is one of the big questions people have about a set like this: when you're trying to have video and audio working together, how flexible can you be with the audio? How much can you deviate from the particular script?
Quite a lot, actually. Maybe in the next track it'll be a bit more obvious—it's way simpler than this one. The question is not so much how far can you modify what you're doing; the question is, how much do you want to? Or how far are you able to do that? Since I started to play music live, I realized that there's a certain balance between how much you could do, and how much you really will do, from experience. From a very early moment I used very complex machines onstage, and you never really use the entire complexity it gives you. You always end up with a certain number of parameters you want to tweak, and after realizing that it's much more inspiring for me to have far simpler machines onstage and just focus on a couple of parameters you want to tweak.
So a track like this, for example—you realize when playing it what's possible and what's not possible. You thought [some aspect of performance] would be fun while preparing it, but it's not actually that entertaining when you're doing it. So this is how certain ideas ended. Basically the whole experience for me shrinks down to a very defined choreography, I would say. You know what you like about a certain track, you know what works and what doesn't. And within the whole set, which is a one-hour set, I have to find the spots to improvise and the spots where it's not necessary to improvise. So like I said, the first six minutes of this track, I wouldn't have to be onstage, actually—I could play and just remain onstage fixing the audio and doing little things here and there. Then it goes into a couple of loops after six minutes. It gets stuck in a loop, and I can improvise on top of that, and audio from the MPC is then layered on top, and so on. And then I play some sounds, which trigger some video. The very end section—the whole track, if I just run through it and don't really improvise, it would be maybe seven minutes, but I've played that track for maybe 25 minutes. If the vibe is good and the sound is good and I've realised that people want more, or it's just fun to do when I have the time, then I can expand it quite a lot.
I imagine that's the important part of doing a set like this—if it's too planned, you can't really react to the crowd. You're basically just getting up and pressing play, and giving people the same thing they'd get if they saw you on any other day. But at the same time, you also don't want it to be too improvisational or too open, or else you probably wouldn't be able to have such sophisticated things going on with the video and all of that. Is it a balance?
I think so, yeah. There are a couple of factors here that you have to start taking into account before making a set. For example, before performing at a festival such as MUTEK. It's a very different situation from performing at a sort of concert, where they invite just you to play. You could play for two hours if you wanted. At a festival you're given a certain slot, and they tell you to play for 40 minutes or whatever, and then someone comes on and says, "That's it." So you can't really improvise entirely, even though you wanted to, and you even sometimes have to take bits and pieces out of the longer set to make a shorter set.
To me it's rather important to be able to structure the entire thing, time-wise, to have an idea for how long it can be and develop a feeling for it. I'm not looking at the clock when I play; I have a feeling for it. I know, if I improvise at a certain time, there are some islands of safety after the improvisation part where I can relax a little, where it's pretty structured and I don't have to worry. I personally need that structure for that kind of set, because it's really, really complex. It's a very simple setup, but this means it's really, really complex, the way to program it—like how its interlinked, how the media and the audio are different. If you make a small mistake, like switching the wrong program at the wrong moment, you can mess it up. So there's a lot of concentration involved, and making it 100% improvisation would be a bit too much to focus on.
You mentioned that there are moments where things could malfunction and could go wrong—
And sometimes they do, and I have no idea why [laughs].
Does that happen often? And what do you do in that situation?
You act as if nothing has happened [laughs]. Yeah, it sometimes happens. It's quite mysterious actually. Every now and then—I would say maybe ten percent of shows—there's a glitch—sometimes a major glitch, sometimes a minor glitch—where the program wouldn't change, for example. And normally I rehearse that; for the technical side of things, I rehearse before I go on tour. If I change a piece of equipment, or a cable even, I run the whole set for a couple of days, actually, emulating the live show. And it's happened quite a lot that it's all identical: the cables are numbered, it's always the same connections, I never change the cables, it's always the same thing, and it suddenly doesn't trigger and I don't know why. I make a note of it, and when I get home I go back to that glitch, as I want to see what's going on, and then it works. It's weird. This setup is quite, how can I say—I thought a lot about this setup, and the reliability of these machines, and I test them a lot and know what could happen and what the possible bugs could be.
In general I'm pretty prepared, you know? I'm not freaking out like, "What's happening?!" It's always just, "Oh." There are always moments in the structure, and there's moments like this in the whole set: I start the MPC's pattern base, and I start with pattern one. Then I run from pattern to pattern, using the foot pedals—I step through. I never go back from 20 to 16 or something, it's all a sequence. And there are always points in that sequence of patterns where I have programmed a reset. So if you're lucky, you're back on track.
So there's a little bit of troubleshooting that's built into the set up?
As much as possible, yeah.
You've been playing this set now for about two years, right?
Yeah, two and a half years.
How has it developed over time? Or are you basically doing what you were doing before?
No, actually it has. Like I said before, it took me a long time to program it and to make it run, and while on tour I can't really change anything, because it's too complex. So what happened is, I prepared the first version of the set, and then I realised that certain things would not work, or I preferred a different order, or I enjoy a lot of things more than others. So I did a second version a year after where I changed things and took pieces out and put other pieces in and streamlined it a little bit, I would say. And that's basically still the version I'm playing.
I definitely wanted to ask about is this guy [points to spinning Atom™ head on the screen]. We didn't see it in this portion of the set, but there are parts of HD Live where he'll talk or sing along with the music. What are we working with here? He's a computer-generated version of you?
How was he created? Did you film yourself?
Well, there's no 3D rendering or anything involved; it's playing with Adobe After Effects and stuff like that. This is just the picture, a photo. And there's the other parts where I'm singing. And actually, the thing for me performing the HD album live is that there are vocals in there. I didn't want to perform vocals live—I'm not a vocalist—but I wanted that connection to happen, that when the extra voice was sounding, it's not just coming from somewhere but there's some relationship. You see that there's something happening with the vocals, something's happening or someone's singing or something. So my idea was more to create a virtual me, basically, that's singing from the screen. The inspiration was Max Headroom, actually.
So when I started to produce [the set], I had to produce the videos and the audio. The audio was really simple—I had done that hundreds of times before. As for the video I had no idea what that meant. So I bought the video computer, and I bought the software, and then I had to learn After Effects from scratch and Premiere from scratch, and I had to learn blue-screening from scratch. I did the whole thing without anybody else helping me. One of the very last things I had to do was actually the head that's spinning and talking. I wanted it to look really artificial, but I had no idea how to do that. It was just like, "OK, this must be possible, but I don't know how." So I said to my daughter—she was 15 back then—I said, "I have to learn the software. It's called Adobe After Effects, and I need to learn blue-screening, and I have no idea how to do that." Then she said, "Why don't you go on YouTube and look for a tutorial?" [Laughs] I was like, "Yeah, of course!" So I learnt all of the blue screening stuff on YouTube.
That's something I hear all the time from people just talking about audio: if you don't know how to use something, you just go on YouTube.
Exactly. And interestingly, when I wanted to start looking into blue-screening, I thought it would require a small team of people: somebody with a camera, somebody doing the lights, somebody even for makeup. It's a small anecdote, but I was thinking about how to do that. I was about to buy light equipment and a blue screen. I was standing in my bathroom, which has this huge open roof, in the summertime in Chile, so it's really, really sunny, and there's this huge mirror. I was shaving, and behind me there's a thing to hang a towel. And I'm standing there, looking at myself. The problem with blue-screening was, how would I film myself, and how would I see what I'm filming without anybody else doing it? And I realised that I was looking at the mirror, and actually the light was perfect. I could hang the blue screen behind me where the towels go [laughs].
That's how I did it, basically: I could put the camera in front of the mirror, so I could see in the mirror myself, in film. And the light was perfect, and I could put a grey towel behind me, so I wouldn't even have to buy the blue screen! But I wanted to see if it worked, you know? And it worked. I just filmed myself and blue-screened, and I cut myself out and did all the Adobe work. But to my surprise, something I had not contemplated before I started was the fact that the video was way slower and, data-wise, a very, very big task. When you're making an album, even if it's complex work, the session data and stuff is a couple of gigabytes of production data, you know? Then I suddenly realised that just filming myself was like four gigabytes. And making a version of that would be eight gigabytes. So suddenly I had, like, hundreds of gigabytes. I wasn't prepared for managing the amount of data, copying things, rendering things. It started to slow me down. The further I got into production, the more data I had, and the slower it got. Then the tour came up. I had the tour already planned and booked and everything, and the video actually kind of threw me off the plan, and I couldn't really finish all the tracks. I used them when I put together the second version of the set.
How much hard disk space does a show like this set you back?
On the video computer actually, on the day that it's being played it's not that big anymore.
Because it's already been rendered out?
Exactly, so I don't know—like 60 gigabytes or something. The video computer is not even a new machine, it's a four year old Mac. It's ok, it's enough. But also, I'd planned to buy new software for playing back the video, but I had no idea how that would behave when I configured it with MIDI. So I bought it, because I wanted the maintenance, the troubleshooting—being able to call the people and being like, "I had a problem" or whatever. So I bought and installed it and when I started to put the set together suddenly it would crash and I wouldn't know why or what the problem was. Was it the codec, or the amount of data? So there was actually a lot of time spent troubleshooting with that and playing it over and over and over again, until I had the feeling it was kind of stable. But it was a lot of trial and error and re-rendering the whole thing in a different codec, finding the right proportion between the codecs, compression and a lot of data, and so on and so on. In general, triggering with HD video off a laptop is the tricky thing with that software. It never feels 100% safe.
I know one of the big technical considerations of this set is getting the video to the projector and then to the screen without a huge amount of lag. It's more complicated than just plugging an HDMI cable into the projector. What do you have to do to make sure that what's here on the laptop is matching up with what's there on the screen?
There's a funny story. I went on tour in France last year—it was like seven shows in a row, every day a different show, and the guy who booked the tour for me also was the tour manager. And he asked me before, he said, "Tell me, what do I have to know about your setup, so I can help you? Is the video very difficult?" And I said, "Well, actually you will see that every single day there will be a different problem with the video." And he said, "No, it's not possible. It's a standard. It's HDMI, it's just—you plug it in, and..." And I said, "You will see." [Laughs]
I have a couple of things in my tech rider, it's quite specific what I need: it's an amplifier, or an Ethernet transmission system. There's not too many options. And then, on the first day, the projector would not recognise the computer. On the second day, it would be noise somewhere, and so on and so on. Every day there was a different issue, and every day it took me like two hours, being patient, talking to the people. And it's always the same routine, because the video technician always says, "Oh, it's your computer." And I'm always like, "No, it's not my computer, I know it's not my computer." [Laughs.] And then you have to go through it—they go get another cable, and I don't know, it's sometimes the adapter, sometimes the cable or the combination between the two. It's quite mysterious, actually. It's not as standard as you would think it is. And then every now and then you have huge latency, where it's so off that it's not fun to play. Then I have to delay the audio on the main board, which makes it really hard to improvise because you're like half a millisecond off. It's a lot of concentration and trying to have fun onstage, but its really not comfortable, not smooth. All that kind of stuff can happen and you have to be prepared for that. And be patient.
You've been doing this set for a number of years now. Are you beginning to think of what the next step is? Do you have some ideas for new ways that you could perform onstage?
Yes, but it's not very concrete just yet. This is basically the last year I will perform this set, and then I have to go on to something else. I have a couple of ideas, musically and visually, of what that could be, but I'm not very far with that just yet.
As you're working on new music in the studio, now that you have all of this experience with video, does that influence the music making process at all?
No. Whenever I make music in the studio, I know if I would like to play that live or not and in which way. Sometimes I think it's not always necessary to have video. There's a couple of sets, like the one I play with Tobias Freund, for example, without any visuals, and it's about a totally different idea; it's really more about sound than anything else. And, yeah, whenever I make music I already have the feeling for whether it can be played live. Not all the music I make should be played live. There's different moments in music where it's better to just listen to, you know—at home, for example. And not all music needs the visuals, either. For example, the other piece I'd like to play used to be an encore for the first A/V set. It was a piece I had just thrown in very quickly, just in case people wanted me to play more. It's just out of the MPC, and [the visuals are] basically just a very small JPEG, a very simple black and white JPEG. And what I do is, with the MPC, I play plug-ins on top of that JPEG, and the overload of the different plug-ins does all the effects. And it was so much fun to play that I put it in the set and kicked something else out. It's actually one of the only old pieces I had to transfer into the new system, so it's an entirely different idea, and it's a piece of music that very much only exists because it's so much fun to play with video. It wouldn't be half as much fun without it.