Vision Pros

Transcript of Interview

James Swiney

(2m 7s) Tim Chaten:

Welcome to the podcast, James.

(2m 9s) James Swiney:

Thanks so much for having me.

(2m 10s) Tim Chaten:

Absolutely, I’m excited for the chat ’cause you are the first game developer we’ve had on and gaming, I think, could be pretty big in this new Vision Pro platform here.

(2m 22s) James Swiney:

Yeah, I really hope so. Yeah, I’m really looking forward to it.

(2m 25s) Tim Chaten:

Yeah, so before we get too deep into things,

(2m 28s)

I’m just curious, what’s your experience,

(2m 32s)

do you have any VR headset experience outside of working on the Apple Vision Pro?

(2m 37s)

Like, do you have an Oculus or a PlayStation VR or what kind of experience do you have prior?

(2m 43s) James Swiney:

I bought the PlayStation VR 2 when it first came out and yeah, I really, I really like that.

(2m 53s)

And as you’ll note by some of the work I’m doing on the Vision Pro, one of my favorite games on the PS VR 2 is Tetris Effect.

(3m 4s) Tim Chaten:

Oh, that is such, that’s an amazing game.

(3m 8s)

Like, I just want to sidetrack here.

(3m 11s)

Just like, it’s kind of like a meditative experience where you come out feeling like a better person,

(3m 17s)

but you can also beat your head against the wall with some insanely hard Tetris levels at the same time.

(3m 21s)

It’s a weird dichotomy there.

(3m 25s) James Swiney:

Yeah, yeah, it’s incredible and it’s so good at getting you in the zone that you just totally forget where you are or anything.

(3m 31s)

Yeah, so yeah, I really like the PSVR too.

(3m 37s)

I had to work up over time.

(3m 40s)

I got some motion sickness when I first got it.

(3m 43s)

I tried to play Gran Turismo straight away and had to work up slowly.

(3m 48s)

But now I’m good.

(3m 50s)

I can wear it for a long time and play anything.

(3m 53s) Tim Chaten:

That’s awesome. Did you go down the dark rabbit hole of getting a little racing rig for the Grinchy’s Turismo yet?

(4m 1s) James Swiney:

I had them on Amazon in my cart, but I never pulled the trigger.

(4m 6s) Tim Chaten:

Okay. I should say it’s well worth it. I have one. Yes. Yeah. Yeah. And as far as the motion signals, I got one of those little bracelet things that like sends electrical signals through your body?

(4m 6s) James Swiney:

Yeah, so it would be very cool.

(4m 10s)

Oh, nice.

Reliefband

(4m 20s) James Swiney:

Oh cool, I haven’t heard of that.

(4m 20s) Tim Chaten:

Yeah, um…

(4m 23s)

The Relief Band.

(4m 25s)

So, I normally don’t need it, but if I’m like,

(4m 29s)

“Oh, I’m gonna be doing this super intensive VR game, let’s throw this on.”

(4m 33s)

Just like, let me run around, like in Synapse, where I’m running and like,

(4m 38s)

just doing crazy stuff.

(4m 40s)

It’s nice to have that added, like, it’s just like…

(4m 43s)

I guess the electricity, like, kind of ground your body in a way that’s like,

(4m 47s)

“Oh, you’re good.”

(4m 51s)

But anyways.

(4m 53s)

Yeah, PS Vita 2 is great and it’s kind of like imagining Apple Vision Pro.

(4m 58s)

Oh, the display is going to be twice as good as this already great display in here.

(5m 5s) James Swiney:

Yeah, yeah, that’s one thing that’s got me really excited.

(5m 10s)

When you watch the videos of how easy it is to put on and off in all the videos,

(5m 15s)

compared to how cumbersome the PSVR 2 is to put on and off.

(5m 20s)

One of the reasons that usually halts me from getting out the PSVR is just how cumbersome it is to plug it in and then you’ve got to set it up.

(5m 29s)

And then I always have to go in and adjust the eye levels and all that sort of stuff.

(5m 32s) Tim Chaten:

Oh, you do. Interesting. They stay locked in pretty well.

(5m 35s) James Swiney:

Oh, nice.

(5m 37s) Tim Chaten:

It’s funny you say the PSVR 2 is cumbersome because I also have a PSVR 1.

(5m 44s)

I’m not sure if you remember that one. It had a breakout box and the two HDMI cables coming from the headset.

(5m 44s) James Swiney:

Yeah.

(5m 45s)

Yeah.

(5m 51s) Tim Chaten:

That was a whole other thing. You had the external camera to make sure that was up and running.

(5m 56s) James Swiney:

Oh yeah, yeah I’m sure.

(5m 57s) Tim Chaten:

But, yeah.

(6m 2s)

As far as, so back to Tetris Effect, that game is kind of interesting because it works with the sense controllers,

(6m 8s)

but also just works with the standard dual sense.

(6m 10s)

And the sense controllers are nice having your hands separate and stuff, but Tetris is a game that, you know, it just works.

(6m 17s)

You know, no matter what kind of your interactive method is.

(6m 21s)

With Vision Pro, what are your thoughts of just, you know,

Dedicated VR controllers

(6m 27s) Tim Chaten:

using, you know, dual sense controllers and Xbox controllers.

(6m 32s)

vs. these dedicated hand controllers that other headsets have had.

(6m 41s) James Swiney:

Yeah, well I think it’s great. I think with the Vision Pro, you can tell by Apple’s marketing they’re not really leaning towards the fully immersive gaming style. So something like Synapse or something like that, they don’t appear to be that interested in. But that’s where I bring up Tetris Effect, because Tetris Effect is the kind of thing that could definitely work on the Vision Pro, even with sort of

(7m 11s)

are. You could have the Tetris board in a window and have the effects in your environment around you, which is kind of what I’m doing as well. And using a DualSense remote, I’m really glad that works with the Vision Pro because it’s a really easy thing to pick up. And because you’ve got the pass-through, you can see what you’re picking up. There’s not

(7m 34s)

issues with any of that sort of stuff. And yeah, I think having

(7m 41s)

both type of control schemes is what’s going to be similar, is what Apple’s going to want. But yeah, I definitely think supporting game controllers, and since it’s so easy with the game controller framework, yeah, I think it’s a no-brainer to support both. But also having the DualSense in there is a really cool way to play in VR.

(8m 3s) Tim Chaten:

Yeah, for sure. Yeah, I hope long-term, you know, a couple of OS’s from now, you know, we get some Apple-made

(8m 12s)

handheld controllers because

(8m 14s)

You can do the hand tracking fine, but there’s something about haptics on your hands as you’re doing stuff in a game

(8m 21s)

even more so than just

(8m 23s)

Having the buttons which buttons are also really handy to have in the fully immersive stuff. So I hope we get there at some point

(8m 31s) James Swiney:

Yeah, I could see Apple doing something, again, different from everyone else, but still something very interesting.

(8m 35s) Tim Chaten:

A glove.

(8m 37s) James Swiney:

Yeah.

(8m 38s)

Yeah.

(8m 39s)

Something.

(8m 40s)

Oh, yeah.

(8m 41s)

Something quite small that you heard, but still provides feedback and control.

(8m 44s) Tim Chaten:

Yeah, I am.

(8m 46s)

I am curious if Tetris Effect will come to Vision Pro.

(8m 50s)

That does seem like a no-brainer to me.

(8m 51s)

They’re on other platforms like Switch and I think Xbox.

(8m 55s) James Swiney:

Yeah, it’s definitely a very cool game that can work in both AR and VR.

(9m 3s)

One of the other games that I have on PlayStation VR 2 that I know they’re working on a Vision Pro version is DEMIO,

(9m 12s)

which is a sort of a board game, and I’ve seen demos, I think, for the Quest of their AR version,

(9m 20s)

where you kind of place a game board on a table in…

(9m 25s)

the physical world, and then the game board has all the animated characters on it that you can still pick up and move around,

(9m 33s)

which is the coolest thing ever.

(9m 35s)

As someone that plays board games with figurines, having one with animated ones in front of you is just like the dream.

(9m 42s) Tim Chaten:

Yeah, like that will be interesting like a clear off your table in your living room so you can play

(9m 49s)

And yeah, it’d be cool if they did local multiplayer and if you’re fortunate enough to have two headsets of the family

(9m 59s) James Swiney:

Yeah, that’s it. That’s a bit of a stretch, but yeah.

(9m 59s) Tim Chaten:

I don’t know

(10m 1s)

Right, I mean long term like in ten years

(10m 5s)

eventually, you’ll probably have more than one of these things in your house and

(10m 9s)

Hopefully it’ll work with older headsets, so you don’t have to buy it

(10m 12s)

the current gen a couple of times over, yeah.

Void X

(10m 18s) Tim Chaten:

So you’re bringing VoidX to Vision Pro, that’s kind of your current project for Vision Pro.

(10m 24s)

Can you share a bit about what this game is?

(10m 27s) James Swiney:

Yeah, sure. So this is a game I originally released on the iPhone last year.

(10m 34s)

And I’ve made a few of them over the years. One of my favorite genres in gaming is shoot ’em ups.

(10m 43s)

And so VoidX is an arcade-style shoot ’em up. There’s no other progression other than you’ve got to try and beat the boss and get a high score.

(10m 52s)

and you have a limited number of lives.

(10m 54s)

And I…

(10m 57s)

I ended up getting some really cool art from a pixel artist for all the ships and I went with a style of the basically monochrome with only sort of red and green as the indicators of your damage versus the enemies.

(11m 16s)

And I ended up with a kind of a bit of a unique style for it that I’m really happy with and I released it and it actually did quite well.

(11m 27s)

I was lucky enough to have Apple feature it under their best new game section and I then went on and released it on the Apple Watch as well.

(11m 40s)

I was actually inspired by someone who open sourced a version of Quake on the Apple Watch and I thought to myself, if someone could put Quake on the Apple Watch, surely I can put my little shoot ’em up on that. So I gave it a go.

(11m 57s)

And yeah, I managed to get it running and I even released it and a couple of websites picked up on that too, which gave me a couple more installs, which was nice.

(12m 5s) Tim Chaten:

Yeah, I’ve got it installed on my Apple Watch and it’s a beautiful rendition there. The art style, it’s perfect. The black background and you can see everything nice and clear even on the regular series watches, not even the Ultra. It runs and looks great there.

(12m 29s)

Is sound possible on the Apple Watch through the speakers?

(12m 35s)

I don’t need AirPods for sound to work. Okay. Gotcha. No, that’s fair.

(12m 39s) James Swiney:

To be honest, I didn’t even attempt it.

(12m 41s)

I completely let out the sound engine on the Apple Watch.

(12m 46s)

I wasn’t sure anyone would actually want it playing through the speaker of the Apple Watch.

(12m 53s)

So I didn’t even try. I’m not sure, actually.

(12m 58s) Tim Chaten:

Yeah. No, it’s funny. My watch is mostly just always in mute and then I’ll unmute to see if anything happens if I do that.

(13m 5s)

[laughs]

(13m 6s)

Yeah.

(13m 8s)

And the crown haptics are rather nice there.

(13m 11s)

So this will be on Apple Watch all the way up to Division Pro in this ginormous kind of setup.

(13m 17s)

And as you said, yeah, the screenshots I’ve seen do remind me of Tetris Effect where you’re kind of

(13m 25s)

almost sitting in the environment and you’re able to kind of see like lightning around you and in Tetris Effect you have dolphins swimming next to you and stuff.

Room effects

(13m 35s) Tim Chaten:

• What kind of effects can we expect from the Vision Pro version?

(13m 39s)

You know, lightning and when the boss blows up,

(13m 44s) James Swiney:

Yeah, so it was kind of a progressive thing that I was going through when the simulator first came out.

(13m 52s)

Obviously the first thing I did was try a bunch of my projects and see how they went in the simulator.

(13m 59s)

And when I got VoidX in there, I started playing around with the Reality Kit framework and I found that they had the particle emitters that you could put in there.

(14m 12s)

That’s how I was doing the rain.

(14m 14s)

in Voidx Aurelii, that’s a particle emitter in SpriteKit, so I thought why don’t I just use the particle sprites that I already have and see how it looks in the simulator. And I did the rain

(14m 27s)

and did a quick demo of the rain surrounding the window that you’re playing in. I thought oh this is really cool and then I just added some enemies flying by and yeah it kind of went from there. I remember I posted it on Twitter the very first one with the rain.

(14m 44s)

I immediately got a comment from someone that said “you’ve got to put that lightning in there”

(14m 49s)

and so I went to work and found out that yep the particles supported animations so

(14m 55s)

I got the lightning and moved from there. The bullets coming out of the boss

(15m 2s)

was definitely inspired by things like Tetris Effect and that sort of stuff. I just thought wouldn’t it be cool if you’re playing in a normal 2D window but the effects are coming

(15m 14s)

while you’re playing which is something completely unique to VR and especially the Vision Pro.

(15m 22s)

I really like the idea of the window is floating in your normal physical space but the rain is coming from the roof of your house and the particles are coming out of the window that you’re playing in. It was a really fun thing playing around with and the more I added to it

(15m 41s)

the cooler, right? The cooler I thought.

AR Games

(15m 46s) Tim Chaten:

Have a 3ds from Nintendo back in the day

(15m 50s) James Swiney:

No, I’ve never owned one of those. In fact, the first Nintendo console I’ve ever owned is Switch, so yeah.

(15m 57s) Tim Chaten:

That’s amazing. That is amazing. Okay. I was gonna mention they had this whole thing in the early days of

(16m 6s)

augmented reality they had these 3d cameras on the back and

(16m 10s)

front I believe just maybe this back but you’d like scan different cars and put things in your environment, but

(16m 18s)

your environment became a

(16m 21s)

Part of the game almost where you’d be these endless shooters and things coming at you and you’re trying to shoot them down and stuff

(16m 27s)

Can you imagine?

(16m 29s)

Like fully AR games within vision Pro like you’re looking at things and using your eye

(16m 37s)

You probably can’t use your eyes to aim. Is that an accurate statement?

(16m 42s) James Swiney:

So, Apple released a demo game as like a sample project.

(16m 55s)

And the way you do it in that game is you actually make a love heart with your hands

(17m)

and you shoot love hearts at enemies that are in the room, like at you as well.

(17m 7s)

So, yeah, that’s an interesting…

(17m 9s)

I think it was more to demo sort of the hand tracking and that sort of stuff as well.

(17m 12s)

But yeah, it’s something that you could definitely have.

(17m 19s)

I, to be honest, I’m not actually that interested in the fully augmented reality sort of 3D stuff.

(17m 31s)

Apple really interested me with the spatial computing angle that they went with.

(17m 38s)

And that’s why, so I’ve got the, you’ve got the PSVR and you’ve got the…

(17m 42s)

the Quest and that sort of stuff for the fully immersive stuff.

(17m 46s)

But Apple’s definitely not leaning that angle.

(17m 49s)

And I do like the idea of…

(17m 53s)

of mixing the two and having the sort of the floating windows that you play games in as well.

(17m 59s)

And I also, I think to begin with,

(18m 4s)

we’ll see a lot of developers just bringing sort of the 2D playing stuff.

(18m 7s)

And what I’d like to see is people instead of trying to rethink like

(18m 12s)

a 3D interface, still embracing this, like the 2D windows,

(18m 16s)

but perhaps having effects that come out of the window while you’re using it.

(18m 21s)

So like a simple weather app or something,

(18m 24s)

it maybe has clouds that form over the top of the window or something like that.

(18m 28s)

Just small things, but they make a big difference

(18m 31s)

in how you’re using them sort of around you.

(18m 35s)

And that way also you can lean into having multiple things open at the same time.

Multitasking?

(18m 41s) Tim Chaten:

If you glance at a different window, say you have your game open and another window open,

(18m 48s)

if you look over the other window, do your effects, your 3D effects,

(18m 54s)

do they go away? Or how does that work?

(18m 57s) James Swiney:

Okay, so for Void-X, it has to be the only one open.

(19m 1s)

So it’s using an immersive scene,

(19m 5s)

which means it is the only thing that’s open,

(19m 8s)

even though you can see your surroundings.

(19m 11s)

It’s an immersive scene, but it’s set to mixed mode,

(19m 14s)

and mixed mode means you see your surroundings,

(19m 16s)

but you’re in the whole scene.

(19m 19s)

And that’s how I can actually anchor the particle effects

(19m 27s)

and the boss’s position on the window.

(19m 29s)

I originally had it as a normal window,

(19m 32s)

which spawned a scene around it,

(19m 34s)

so you could toggle that on and off.

(19m 37s)

The problem I had was there was no way I could figure out

(19m 40s)

how to position particles in the right spot,

(19m 43s)

especially if the user moved the window.

(19m 45s)

I couldn’t do that.

(19m 46s)

So instead, I have an immersive scene,

(19m 49s)

and the game window is what’s called an attachment in that scene.

(19m 55s)

So it’s an entity.

(19m 57s)

in the scene so I can position it and anchor things around it how I want.

(20m 2s)

And then I also added a mode where you can,

(20m 9s)

there’s a button that switches modes and will switch to a fully immersive scene,

(20m 14s)

which surround,

(20m 15s)

like blacks out the room and surrounds the user with the cityscape.

(20m 19s)

And it also gets rid of the background on the game window.

(20m 23s)

so the ships kind of appear floating in space.

(20m 27s)

That’s one of the ones that’s really hard to describe in a screenshot.

(20m 29s) Tim Chaten:

Is that the third screenshot you sent me?

(20m 32s)

Okay, excellent. Yeah, I was it was mm-hmm

(20m 40s) James Swiney:

It really needs to be a video or something like that.

(20m 43s)

And even in the simulator, it kind of looks a bit weird because the depth of it is hard to show off, I guess,

(20m 52s)

in a 2D screenshot or video.

(20m 55s) Tim Chaten:

Have you played around with replacing your floor with kind of the floor from the game, stuff like that?

(21m 1s) James Swiney:

Not too far. I haven’t actually gone, I haven’t leaned too hard into this one.

(21m 7s)

That was kind of a mock-up that I did quickly that turned out pretty good.

(21m 12s)

And so I decided to keep it as a mode.

(21m 14s)

But yeah, I definitely haven’t leaned too hard into the fully immersed thing.

(21m 20s)

I’ve mainly focused on the sort of AR.

(21m 23s) Tim Chaten:

Yeah, it’d be it’d be kind of funny probably a lot of work. I’m not sure how

(21m 28s)

This would work if if you have like a lamp or a table in your room if you have like the artist

(21m 36s)

Draw up what a lamp or table would look like in your world and like replace

(21m 41s)

objects in your you know real life with

(21m 45s)

You know a similar thing for that art style. That would just be a fun thing. Yeah

(21m 45s) James Swiney:

uh, with objects. That would be pretty cool, yes.

(21m 50s) Tim Chaten:

But yeah, I’m not sure how well there is is there

(21m 53s)

is object object detection within Vision OS.

(21m 57s) James Swiney:

Yeah, there is. Yeah, there is. That’s again something I haven’t really played around with.

(22m 3s) Tim Chaten:

Environmental effects like this would work within multitasking if you weren’t doing things like exploding from the actual screen more like ambiently, “Here’s lightning.”

(22m 15s) James Swiney:

Yeah, so if you use a volume or something like that,

(22m 22s)

then you can have that with other windows of other apps and that sort of thing.

(22m 26s)

But as soon as you spawn an immersive space,

(22m 29s)

everything else fades away and you’re just in that.

(22m 32s)

So yeah, definitely when I was talking about the weather using a Cloud or something,

(22m 35s)

they could use a volume which would let them have 3D effects in the same space as a window kind of thing.

The Boss

(22m 44s) Tim Chaten:

Noah B, I was just thinking about this, pretty cool, is if when the boss shows up, you could have the boss fly in from behind your shoulder into the screen.

(22m 54s)

I don’t know, stuff like that just made me think, like, just other things you could do that still use the 2D plane, but kind of like put more panaz on it.

(23m 4s) James Swiney:

Yeah, so I’ve also been experimenting with having billboards that appear up behind the current window that could maybe flash up high scores or as giant versions of the bosses that come in or explode or something up behind.

(23m 19s)

Yeah, so I’ve definitely been playing around with more than just what I have.

(23m 26s)

But also, I’m trying to keep it also as simple as possible.

(23m 32s)

so it ends up…

(23m 34s)

I kind of want the first time someone gets to one of those bosses, they don’t expect the bullets to come out of the screen.

(23m 42s)

They just… the boss comes on, starts shooting, and then all of a sudden there’s bullets coming towards them.

(23m 47s)

It’s kind of a bit of a wow effect in the game, yeah.

(23m 51s) Tim Chaten:

How many bosses are there?

(23m 53s)

No, they’re kind of different styles.

(23m 55s) James Swiney:

In the game, there are nine bosses altogether.

(24m)

Um, and I’ve had to actually.

(24m 4s)

Um, so I’ve definitely so far on the vision pro. Uh, what I’ve decided to do is, uh, reduce the difficulty of the game because I don’t know if you played on the iPhone. It’s quite difficult. Um, and so on the vision pro. I wanted to be more of an experience than a sort of hardcore shooter that they’re trying to play. So I’ve definitely reduced the difficulty reduced. How many bullets reduced? How many enemies and that sort of stuff? Because I like people to actually experience.

(24m 30s)

bosses as they go through as opposed to just hitting the first boss and dying over and over.

(24m 35s) Tim Chaten:

Yes.

(24m 36s)

Is an art gallery of the bosses something you’ve explored as like, that could be a fun little bonus in Vision Pro to see bigger renditions?

(24m 49s) James Swiney:

Yeah, that is a good idea actually. Yeah, that is a good idea actually, of, yeah, just letting someone pick a boss and

(24m 55s)

and sit there and watch it or whatever. Yeah.

(24m 57s) Tim Chaten:

Yeah. What are some of your favorite bullet hell shooters?

Favorite Bullet Hell Shooters?

(25m 2s) Tim Chaten:

Like I’m thinking of Rezogun and what are some of…

(25m 6s) James Swiney:

So most of the bullet hell shooters that I’ve played are in the, I never know how to pronounce it properly, but the Taohao series. So T-O-U, H-O-U series. Yeah, I’ve played quite a lot of those.

(25m 21s)

And also on iOS, I absolutely love the Danmaku Unlimited games. They’re absolutely fantastic bullet hell shooters. And then, yeah, my like for shooters go right back to the

(25m 36s)

Sega Master System, where I played Astro Warrior and also R-Type and those sort of games. Yeah,

(25m 43s)

I’ve been a fan of them for a long time. And one of my first

(25m 50s)

quite games that did pretty well on the Apple Store were a game called Hectic Space that I released, I think it’s about six to seven years ago. It came out around the same time as sort of the Flappy Bird craze and that sort of stuff.

(26m 7s)

And it again managed to get featured by Apple and did quite well at the time.

(26m 13s) Tim Chaten:

That’s awesome, yeah. As far as controlling this game on Vision OS, you of course have controller support. I’m curious, hand gestures, what’s the thought there? I’d imagine you’d need some way to control this game without a controller if you’re in that situation.

Input

(26m 33s) James Swiney:

Exactly, yes. So it works because the touches translate very well to the Vision Pro.

(26m 43s)

So basically, looking at the window where the game is,

(26m 49s)

you can control that just by using the standard Vision Pro.

(26m 53s)

So like in the simulator, you just drag the mouse.

(26m 55s)

So it’s just the same as dragging.

(26m 57s)

You’ll drag the ship around the screen.

(26m 59s)

The only thing I’ve noticed is I’ve needed to increase the sensitivity.

(27m 3s)

So the less you move, the more the ship moves kind of thing.

(27m 9s)

But yeah, I think both ways will be suitable for controlling it.

(27m 15s)

And I think most of the time, most people won’t want to grab a controller.

(27m 19s)

So supporting the built-in gestures were definitely really important to get it in there.

(27m 23s) Tim Chaten:

So what’s what is the gesture for like going up and down?

(27m 28s) James Swiney:

So it’ll be the same as scrolling.

(27m 30s)

So just scrolling up and down and left and right.

(27m 33s)

Sort of in the demos, you see they pinch their fingers and move up and down, left and right kind of thing.

(27m 37s) Tim Chaten:

• Okay.

(27m 40s)

Have you considered custom gestures?

(27m 41s)

Like if you’re sitting at a table,

(27m 43s)

just like moving your finger up or down,

(27m 46s)

is that possible or something you’ve investigated?

(27m 50s)

Custom gestures as a add-on.

(27m 53s) James Swiney:

No, I haven’t investigated any custom gestures for the Vision Pro.

(27m 58s)

I do understand why Apple has gone with the gestures they’ve gone with.

(28m 5s)

Because if you play any VR game where you have to hold, even shooting games in VR,

(28m 11s)

where you have to hold your hand up for a long period of time, it’s really tiring.

(28m 15s)

And you don’t want to be browsing the web with your finger up in the air or something.

(28m 19s)

Yeah, so…

(28m 24s)

Yeah, I’m definitely, I’m interested to see what people do.

(28m 28s)

But I, yeah, for launch, I’m gonna stick with what’s built in and the stuff I get for free, yeah.

(28m 37s) Tim Chaten:

And to actually shoot on iOS, I believe on iOS, you have to be tapping.

(28m 46s) James Swiney:

So as soon as you move, yeah, as soon as you touch the screen, it shoots. So yeah, it’s, it’s, yeah,

(28m 46s) Tim Chaten:

Yeah.

(28m 47s)

Yeah, yeah.

(28m 48s)

Will this just be constantly shooting?

(28m 52s)

You don’t have to worry or think about that?

(28m 52s) James Swiney:

um, yeah, I think at the moment it’ll just be constantly shooting. Yeah. Um, yeah. Or when you,

(28m 54s) Tim Chaten:

Okay.

(28m 55s)

Excellent.

(28m 59s) James Swiney:

when you do the touch, uh, it’ll shoot. Yeah. It’s just a, um, I know on the watch I do always shooting. Um, yeah. So, which, yeah, it makes it a bit easier. Um, yeah, something.

(29m 7s) Tim Chaten:

Yes, which is smart.

(29m 9s)

[laughs]

(29m 14s)

Okay, do you get haptics or rumble with the external controllers?

Haptics in external controllers?

(29m 19s) Tim Chaten:

Is that something that’s supported in the SDK?

(29m 22s) James Swiney:

They do support it in the SDK, and unfortunately I’ve been a bit lazy and haven’t supported it.

(29m 28s) Tim Chaten:

Okay.

(29m 28s) James Swiney:

I usually jump on game controller framework straight away.

(29m 33s)

I was very quick to support the lighting.

(29m 37s)

In one of my earlier games when you switch swords, I changed the light on the controller to the color of the sword.

(29m 42s)

So I’m using a flame sword, it was red and that sort of stuff.

(29m 45s)

And that’s a good point, I really should get in there and support the haptics in the controller.

(29m 53s)

Because that would definitely add to the game, like when the lightning goes off, the controller should shake.

(29m 58s) Tim Chaten:

And is it just simple Rumble or are you able to do full DualSense?

(30m 3s)

How fully integrated into DualSense does it actually get you?

(30m 7s)

Do you know?

(30m 8s) James Swiney:

I’m not sure. It uses a lot of the same stuff that’s already built into iOS, so I’m pretty sure you can play sounds through the rumble and have it do that effect. So how deep it does that to the DualSense, I’m not sure, but I’m pretty sure you can play a sound effect through it and have the controller then rumble according to what that sound was.

Sponsor: Agenda 18

(30m 33s) Tim Chaten:

This episode is sponsored by Agenda.

(30m 35s)

Agenda, if you haven’t tried it out,

(30m 36s)

is a great note-taking app that really thoughtfully integrates your calendar and reminders into your notes.

(30m 42s)

In this quick break,

(30m 43s)

I wanna share how Agenda integrates with the Reminders app,

(30m 45s)

which, if you haven’t used recently,

(30m 47s)

has come a really long way,

(30m 49s)

and I’m sure will be great on VisionOS with hopefully a fully native Reminders app there when it launches next year.

(30m 56s)

So first off, in the right-sliding panel of Agenda,

(30m 59s)

you’ll see a unified timeline of your calendar

(31m 3s)

and reminders that have a due date.

(31m 4s)

By clicking on that reminder,

(31m 6s)

you’ll be able to act on that reminder in a few different ways.

(31m 8s)

First off, you can create a new note in whatever project you are in that links directly back to that reminder.

(31m 15s)

If that reminder is already linked to a note,

(31m 17s)

that option will be replaced by the ability to jump directly to that note in Agenda.

(31m 22s)

The next option is to add to selected note.

(31m 25s)

So if you’re already working on a note,

(31m 27s)

you can select that to tie that reminder to that note in progress.

(31m 31s)

You can even tie multiple…

(31m 33s)

…reminders to the same note. This is super helpful for larger projects that you may want to have multiple reminders of in the same note.

(31m 40s)

The third option in that sliding panel is the ability to mark the reminder as complete.

(31m 45s)

You can also hit the quick reschedule option to push the reminder back 5 minutes, 15 minutes, 30 minutes, an hour, or 2 hours.

(31m 53s)

You can also push the reminder for next week or next month.

(31m 56s)

These options are all super handy ways of pushing a reminder back quickly rather than manually going in and changing it.

(32m 4s)

If you do really want to change up a reminder, you can do a full edit of that reminder right inside Agenda.

(32m 10s)

This lets you change the title, due date, and you can even change which reminders list it is saved to.

(32m 16s)

Finally, you can open up that reminder in the Reminders app if you want to view that reminder in the context of the larger list…

(32m 23s)

…or even edit the reminder beyond what the Agenda app allows for.

(32m 27s)

The final aspect of integration I want to highlight is that you can simply highlight a line of text…

(32m 33s)

…in an Agenda note and hit Command-Shift-R on your Magic Keyboard to create a new reminder based off that line of text.

(32m 41s)

And that reminder will be linked to that note you are working on.

(32m 44s)

You can also hit Command-Shift-R without any text highlighted to link the entire note to a new reminder.

(32m 50s)

Alternatively, you can hit the plus button on the menu bar at the bottom of the screen…

(32m 54s)

…and select the alarm icon to link the note you are working on to a new reminder.

(32m 59s)

I’ve not seen many other apps integrate with reminders in this deep and thought out way.

(33m 4s)

You can tell the developers behind Agenda use reminders and create this app out of a desire for them to be able to utilize reminders in an even more powerful way.

Using the Apple Vision Pro

(33m 13s) Tim Chaten:

If you haven’t tried Agenda out yet, I’d really encourage you to give it a try.

(33m 17s)

Agenda is free to download and use with no obtrusive ads.

(33m 20s)

To unlock the full power of Agenda, give Agenda Premium a shot.

(33m 24s)

One thing that Agenda continues to do that I love is their approach to premium features and unlocks.

(33m 29s)

If you ever decide to cancel your Agenda Premium subscription, you get to keep it.

(33m 33s)

keep all of the premium features available to you when you were a subscriber, or you can opt for their new Lifetime Unlock, which will unlock the full power of Agenda on all of your Apple devices with a single one-time purchase.

(33m 46s)

To learn more, go to agenda.com.

(33m 48s)

Download Agenda today for free from the App Store on all of your Apple devices.

(33m 53s)

My thanks again to Agenda for sponsoring this episode of Vision Pros.

(33m 56s)

Learn more at http://www.agenda.com.

(34m 1s)

As far as the Vision Pro.

(34m 3s)

How excited are you to get one of these in your own house?

(34m 6s)

And what kind of work do you see yourself doing in the headset?

(34m 9s) James Swiney:

I’m really excited for it. I was a bit bummed that it’s US-only to begin with, so…

(34m 15s) Tim Chaten:

Yes, I’m guessing you’re probably not going to try to import one at an insane cost.

(34m 20s) James Swiney:

So, yeah, I was thinking about what happens if you buy one, and then the other things I thought of is…

(34m 28s)

Like, you see, they don’t need to have an American App Store account, I assume.

(34m 33s) Tim Chaten:

That’s a curious question. Yeah, is the app store only going to be in America?

(34m 34s) James Swiney:

and like I, yeah I assume it would.

(34m 39s) Tim Chaten:

which would be logical, but maybe it’s just I

(34m 43s)

Guess we’ll know when we submit apps what region you’re able to enable vision pro apps for if it’s worldwide

(34m 48s)

Or just where the headset actually

(34m 52s) James Swiney:

Yeah, true. Yeah, so I’m quite curious how that’s going to work because I was, yeah, I worked in a,

(34m 58s)

I worked in an Apple reseller when the iPhone first came out. And even though it was US only,

(35m 5s)

they were everywhere in Australia. But there were so many problems with them. People would constantly upgrade them by accident, which would obviously destroy the jailbreak that they had to use them and yeah.

(35m 18s) Tim Chaten:

Oh yeah, I remember in the early days, in the early days of iPhone, I remember jailbreaking them and then beyond that there was some kind of like radio hack to get it to work on other

(35m 30s)

networks and stuff.

(35m 31s) James Swiney:

Yeah, there was a bunch of hacks you had to do to get it to work in Australia.

(35m 34s)

And if you even did the slightest update that was released, all of those would

(35m 39s) Tim Chaten:

Yes, yep.

(35m 39s) James Swiney:

revert and you would have a very shiny brick in your hand.

(35m 43s)

Yeah.

(35m 45s)

So, yeah, that’s why I went with the 3G when we finally got it in Australia.

(35m 51s)

But yeah, so I would wait for the official Australian version probably.

(35m 56s)

And yeah, I

(35m 59s)

I could see myself trying.

(36m 1s)

to use it. I’m really excited to try this screen mirroring, so using your Mac screen.

(36m 9s)

Because one of the workflows they talked about for developers is you can have your Mac’s Xcode

(36m 18s)

in front of you while you’re working, and then you build from the Xcode and your project builds in the Vision Pro with you alongside that sort of thing. And I think that workflow would be Absolutely awesome.

(36m 31s)

And yeah, I’m just keen to try out having what sort of, I’m really interested to see what people end up doing with spatial computing. Like, do we end up having windows that are all around us or above us? Or do we end up like lying on the couch doing it? Or like, how do we end up sort of doing computing with it? But yeah, I’m very, very excited about it. Yeah, all new Apple products I usually jump on one way or the other.

(37m) Tim Chaten:

Yeah, the spatial computing angles just it was so interesting to me because I’m going into that keynote as like Oh Apple doesn’t get Apple’s I get gaming. I’m a ps4 – for for VR gaming and

(37m 11s)

I mean that side of things it’s sort of right. It’s gonna be games like yours the 2d games that

(37m 19s)

Translate I think best here and then I’ll I’m a ps4 – with the fully immersive stuff for at least a while here

(37m 25s)

But the spatial computing angles like oh, this is a totally different thing. This is a computer

(37m 30s)

And the screens good enough where I can actually read text because I’ve tried reading text in Resident Evil

(37m 36s)

Village on ps4 – and in text is a struggle. It’s there

(37m 38s) James Swiney:

Oh, it’s bad. It is, yeah.

(37m 40s) Tim Chaten:

But it’s a struggle. It’s not the crisp, you know, retina quality that this will be

(37m 45s) James Swiney:

Yeah. Yeah, yeah.

(37m 47s)

And the reports we’ve had that it’s clear and text is very legible.

(37m 53s)

So, yeah, I’m super excited to try using my Mac as a screen in it,

(37m 59s)

but also just using it in general.

(38m 2s)

And also, like, attempting, I do, I do, being an iOS.

(38m 8s)

game developer, I do play a lot of games on my iPhone.

(38m 12s)

And I would be interested to try out a lot of the,

(38m 15s)

a lot of the games as you can scale them quite big,

(38m 19s)

or however is comfortable in there as well.

(38m 22s)

So I think that’d be cool.

(38m 24s)

I even think it’d be cool for simple,

(38m 28s)

simple immersive effects for games.

(38m 30s)

It even could be something Apple could do in sort of terms of like how Hue,

(38m 35s)

you can set up Philips Hue bulbs around your TV.

(38m 38s)

And they will adjust the color according to what the TV is showing.

(38m 45s)

So you can sort of have your wall light up red if the top of the TV is red.

(38m 48s)

So I was thinking, yeah, well, Apple have already got,

(38m 52s)

I think they have the light coming out of your window affects the environment, I think.

(38m 57s)

So yeah, having stuff like that where you could have,

(39m)

you’d be playing something in the room could light up when an explosion happens or that sort of thing.

(39m 6s) Tim Chaten:

Well, imagine you’re the director of The Birds, or he’s not alive anymore, Hitchcock, but imagine the studio behind The Birds, whoever owns that movie.

(39m 16s)

They could create a more fully immersive version of The Birds where if it detects windows,

(39m 22s) James Swiney:

That’d be terrifying, yeah.

(39m 22s) Tim Chaten:

there’s birds attacking in the movie, they attack.

(39m 23s) James Swiney:

[laughs]

Soundscape

(39m 32s) Tim Chaten:

Ed, as far as designing the soundscape within Vision Pro, is it much different?

(39m 39s)

Like, do you have you up till now done any 3D kind of spatial sound for the game?

(39m 46s) James Swiney:

That’s on my to-do list.

(39m 48s)

I want… I’m pretty sure in Reality Kit,

(39m 52s)

you can attach a sound to an entity.

(39m 55s)

So I would probably just attach a sound to each lightning sort of entity.

(40m)

So as it appears that the sound comes from that spot.

(40m 4s)

Currently, I’m still using my same sound engine that’s just on the phone.

(40m 10s)

So I’d probably just remove some sounds from that

(40m 13s)

and position some sounds sort of spatially.

(40m 16s)

But yeah, that’s something I’m yet to look into.

(40m 20s)

Sound is something I always delay because it’s one of the pain points I have with gaming.

(40m 26s)

I always run into trouble.

(40m 27s) Tim Chaten:

In the mobile environment, it’s curious because you have to design your game knowing that many people will probably have this muted, right?

(40m 35s)

Because they might be on a subway or something and they might want to just listen to a podcast while playing the game or something like that.

(40m 42s) James Swiney:

Yeah, yeah. And I learned very early on that when you’re setting up your sound, you have to actually tell it to let them play a podcast. I had a game I released a long time ago when I was first starting out that it had sounds in it, and I didn’t tell it that. So if someone was listening to a podcast and they started my game, it would kill all of their audio just so they could hear my little sound effects. And yeah, that’s not a nice thing to do to

(41m 12s)

Yeah, so for iPhone, I usually just go all in on it. And if they want to listen to it,

(41m 18s)

they can. I always give the option to turn off music. So if they want to listen to a podcast or music, but still listen to sound effects.

(41m 27s) Tim Chaten:

Oh, nice. Yeah, that’s a good combo. I do enjoy that.

(41m 30s)

Yeah. And Gran Turismo, they have a mode where

(41m 35s)

you can actually play Spotify or Apple music

(41m 39s)

and it’ll replace all the music in Gran Turismo with, you know, Spotify podcast if you want from there or, you know, your music.

(41m 46s) James Swiney:

Yeah, I use that all the time. Yeah, it’s great.

(41m 47s) Tim Chaten:

Yeah, it’s great. It’s like this is so smart.

(41m 49s)

It’s keeping all the racing sound effects while keeping

(41m 51s)

while changing the audio mix with that other stuff. Yeah.

(41m 55s)

Yeah.

(41m 55s)

Um…

Releasing an app when you can’t get the hardware?

(41m 57s) Tim Chaten:

As someone working out of Australia,

(42m)

um, do you have any thoughts about releasing a Vision OS app,

(42m 5s)

um, or you can’t easily test the app,

(42m 10s)

um, on hardware?

(42m 13s) James Swiney:

Yeah, I’m really nervous about that, actually.

(42m 17s)

It’s one of those things where I just don’t really know

(42m 21s)

if something’s going to work.

(42m 25s)

I will end up, yeah, I guess I will end up releasing it.

(42m 30s)

I have a test flight that I can send out to people

(42m 32s)

who do have the headsets.

(42m 34s)

But yeah, that again is still not me trying it.

(42m 38s)

And the hardest thing is when someone’s going to report a bug,

(42m 41s)

because they’re definitely going to report a bug.

(42m 43s)

Trying to diagnose that on the simulator is, yeah.

(42m 47s)

And it’s going to be tricky.

(42m 50s)

But when I worked for a company here in Australia

(42m 55s)

that pushed really hard for a watch app

(42m 58s)

when the watch first came out.

(42m 59s)

And yeah, we worked on it for sort of three to four months,

(43m 5s)

and we released a watch app before ever seeing the hardware.

(43m 9s)

And yeah, we pushed it out.

(43m 11s) Tim Chaten:

And the watch, the watch early days was, I’d say probably Apple at its worst as far as new platforms go with that, like using the iPhone as a, the brain.

(43m 24s) James Swiney:

Yeah, it was definitely a rollercoaster ride, but yeah, we got it out and it did work, mostly,

(43m 31s)

which was good, which is the main thing.

(43m 34s)

And yeah, I think you’ve just got to work with what you’ve got and yeah, I’ll just try my best with the simulator and try and get as many people with headsets that I can to actually test it.

(43m 48s)

But then again, I want to try and have it out for the launch of it.

(43m 52s)

So it’s one of the things there that people–

(43m 55s) Tim Chaten:

Yeah, I think people will be, you know, just as iPhone day one, just, I remember downloading all sorts of, well, let me try this. Let me try that. Let me try, you know? Yeah.

(44m 3s) James Swiney:

Yeah, exactly. That’s why it’s one of the best times to have an app out when it’s a brand new device because people will definitely do that.

(44m 12s)

They’ll just go through and they’ll be like, "I’ll try this.

(44m 14s)

I’ll try this. I’ll try this."

(44m 16s)

Whereas if you’re later on, people will overlook.

(44m 20s)

They’re like, "I’ve already got most of the stuff I use.

(44m 22s)

If something really piques my interest, I’ll get it."

(44m 24s)

But early on, and especially,

(44m 26s)

I think the early adopters of Vision Pro,

(44m 27s)

they’ll just be so eager for any content.

(44m 30s)

they’ll be getting everything they possibly can.

Onboarding

(44m 34s) Tim Chaten:

As far as the onboarding process in the game InVision Pro,

(44m 38s)

is there anything kind of special you’re thinking of to like teach them about just using the built-in gestures

(44m 46s)

and if you have a controller,

(44m 47s)

is it gonna be smart enough to tell them,

(44m 49s)

oh, you have a DualSense,

(44m 51s)

here’s this controller layout versus Xbox?

(44m 55s) James Swiney:

Yeah, I’m pretty light on tutorials in my games, mostly.

(44m 59s)

In the game, if you use a controller, it’s pretty simple, you move and shoot.

(45m 5s)

So I usually just expect the user to try it and see what happens, and it will work.

(45m 11s)

I don’t have any tutorials in iOS at all.

(45m 14s)

The user just touches the screen, off they go.

(45m 17s)

On the watch, you’ll notice I have the, I think I say, rotate the crown to move,

(45m 22s)

because that’s not intuitive.

(45m 25s)

And so for the Vision Pro, I’ve just put a thing at the top that says

(45m 33s)

whatever the gesture is to pinch to move or whatever that sort of thing is.

(45m 37s)

So I just have a slight text that fades in and fades out at the top of the screen telling them that they can move with their normal gestures.

(45m 43s) Tim Chaten:

Okay. Yeah, very cool.

(45m 47s) James Swiney:

But it is interesting that you talk about that.

(45m 49s)

I never tell the user they have to look at the screen to move it,

(45m 52s)

but I just assume they would be looking at the screen.

(45m 55s)

Because if they look away from the window, obviously, they no longer have focus on it.

(46m)

So the gestures won’t work, which is an interesting…

(46m 2s) Tim Chaten:

Right. With a fully immersed… your app is fully immersive.

(46m 6s)

Is that… those principles still apply?

(46m 7s) James Swiney:

That’s correct, but…

(46m 10s)

Yeah, so whatever they’re looking at is what will be controlled.

(46m 15s)

So if they’re not looking at the window, I’m not sure if it hangs on to it.

(46m 21s)

Like, if there’s nothing else focused, does it leave to the last one?

(46m 25s)

That’s something I’m not really sure about.

(46m 27s)

But yeah, I definitely have other buttons.

(46m 29s)

So there’s the button above, which is the camera, which switches modes between and that sort of thing as well.

(46m 35s) Tim Chaten:

Okay, and then something I’m actually kind of curious about so your app is fully immersive. Do you know?

Multiple Fully Immersive Apps?

(46m 44s) Tim Chaten:

If I’ve got like four apps open, I’m working and then I

(46m 48s)

Open your app is it kind of like stage manager where it’ll just like put those four apps in a separate kind of stage type thing and then you open your game it takes over and

(46m 59s)

Then like getting back to your old setup of the four other apps

(47m 5s)

that experience is like at this point.

(47m 9s) James Swiney:

Yeah, the simulator just replaces all of the current open apps. I think they just stay how they were and your app fills the screen. But yeah, when you exit, you’re seeing everything’s back to how it was, or you just go back to the home screen with all the icons. And when you dismiss the home screen, all the windows come back again as well. Yeah, so it’s, yeah, it takes over,

(47m 30s)

but it doesn’t dismiss anything, like everything else just stays around. Yeah.

(47m 35s) Tim Chaten:

Because on Vision OS, people say it’s stage manager-esque, but it’s just one stage with unlimited windows.

(47m 43s)

Is that a proper understanding?

(47m 46s) James Swiney:

Yeah, yeah, and you can literally put the windows anywhere, which is pretty crazy.

(47m 47s) Tim Chaten:

Yeah.

(47m 49s)

Okay.

(47m 52s)

Yeah, very cool.

What apps are you most excited to try?

(47m 57s) Tim Chaten:

What apps are you most excited to try out on Vision OS when you eventually get one in

(48m 3s) James Swiney:

I’m really interested, as I said before, to try out the basically just using your Mac as a screen in it as well.

(48m 7s) Tim Chaten:

Yeah, I do hope with future iterations of that idea that you can have Mac, almost like put a Mac app folder in Vision OS and launch individual Mac apps as their own windows.

(48m 9s) James Swiney:

I’m really interested in that.

(48m 25s) Tim Chaten:

That would be very cool versus right now it’s just a mirror of the entire screen.

(48m 30s) James Swiney:

Yeah, yeah, definitely more integration like that would be amazing.

(48m 34s)

As for main apps, obviously playing whatever games I can find that people have ported.

(48m 42s)

And also, just in general, doing the media stuff, like consuming media.

(48m 50s)

But I’m also really interested in trying and seeing how productive the spatial computing is.

(48m 56s)

So sort of like my normal work environment, so having having slack, like as a

(49m)

window, having my work as a window, having like whatever I normally have open in multiple windows around me and see if that I get them in a position that works really well, or if that even does work. Yeah.

(49m 12s) Tim Chaten:

Yeah, I’m super excited to just try working in that environment.

(49m 18s)

It’s going to be fun. It’s like a new way of computing.

(49m 23s)

Even more new than the iPad was. It’s going to be radically different, I think.

(49m 28s) James Swiney:

Yeah, for sure.

(49m 29s) Tim Chaten:

But familiar at the same time, because they’re just a bunch of windows that are TV-sized.

(49m 32s) James Swiney:

Yeah, yeah. And I guess that’s the best thing is any of your existing iPhone apps or iPad apps,

(49m 39s)

unless they specifically don’t let them go on there, they’ll all work.

(49m 45s)

So, and I think that’s where Apple can really excel with the spatial computing.

(49m 51s)

It’s because they already have access to everyone’s iPad apps and everyone’s iPhone apps.

(49m 58s)

We’re not starting from scratch, we’re starting from millions of apps that people already use that you can just use in this new environment.

(50m 5s)

And sure, there won’t be that many at the start that have converted them to full-on Vision Pro apps,

(50m 12s)

but using the iPad versions, from what I’ve tested in the simulator, they all work great.

(50m 19s)

And so, yeah, it’s kind of going to be really good to see those.

(50m 25s)

then I think because I have read a lot.

(50m 28s)

of like a lot of companies like I think Netflix is one of the big ones that came out and said they they won’t support the Vision Pro they’ll just use their they’ll just leave their iPad one on there.

(50m 40s)

Once other people start doing it and once people are using it I think there’ll be more incentive for companies like that but yeah that’s right yeah exactly yeah when everyone says oh look the Disney Plus stuff looks amazing in Vision Pro but don’t worry about Netflix it doesn’t look that great. Yeah.

(50m 42s) Tim Chaten:

It’s like, “Oh, Netflix content kind of doesn’t look great.”

(50m 54s)

Yeah, I already don’t use Netflix that much because they’re in their own silo and don’t integrate with the TV app.

(51m 4s) James Swiney:

Yeah, well, the one thing there, so Netflix are doing really well with me because they’ve got their hooks in. Their game library is slowly growing for iOS and it’s doing really well.

(51m 15s)

And they just announced recently a bunch of good games, including Hades. Yeah, Hades is coming to iOS via Netflix. And I’ve got Hades on every single platform that I have, and I will get get that immediately on my iPhone to play because it’s such a good game.

(51m 19s) Tim Chaten:

There was a huge one they announced, I forget what…

(51m 23s)

Yes, yes.

(51m 35s) James Swiney:

So yeah, I do like the direction Netflix is going with gaming,

(51m 39s)

which, yeah, all their games and stuff should work as well on the Vision Pro.

(51m 47s)

So them just supporting it would be nice.

(51m 50s)

Even just the simplest, being able to watch the Netflix video in the same kind of thing they showed off, like watching it in a huge cinema screen and you’re like…

(52m 1s) Tim Chaten:

I gotta say, there couldn’t be a better time for James Cameron to finally release the 4K restoration of The Abyss,

(52m 7s)

which is coming out in December, that was announced, so that’ll be something I for sure check out in Vision Pro next year.

(52m 15s) James Swiney:

Yeah, definitely. Yeah.

(52m 16s) Tim Chaten:

It would be amazing if they also did some 3D work on that for Vision Pro, but I’m not sure if that will be done.

(52m 23s) James Swiney:

Yeah, yeah. Even just some jellyfish flesh.

(52m 26s) Tim Chaten:

Oh, that’d be very cool, yeah.

SpriteKit

(52m 29s) Tim Chaten:

Well anything else about Vision Pro?

(52m 31s)

that we haven’t covered they’d like to before we wrap it up.

(52m 34s) James Swiney:

So, one thing I thought I’d say is,

(52m 36s)

all my games use native Apple frameworks.

(52m 43s)

So, that’s why it was so easy for me to bring them across to the Vision Pro.

(52m 47s)

But one thing that did have me really nervous,

(52m 50s)

when they first showed off the Vision Pro,

(52m 53s)

they first announced a few frameworks that wouldn’t be supported.

(52m 57s)

And most of my games are written in SpriteKit.

(53m 1s)

and SpriteKit. It’s been around for a while.

(53m 4s)

for a long time. It hasn’t had too much updates from Apple in the recent years,

(53m 11s)

but it still works really well. And it works on every platform super well.

(53m 14s)

So I can run my games on the Apple TV, the watch, the phone,

(53m 19s)

and it’s ported without much effort, which is really great.

(53m 23s)

And it also mixes well with SwiftUI, so you kind of get the best of both worlds.

(53m 28s)

When they first announced the Vision Pro, they said SpriteKit is not supported,

(53m 34s)

heart sank. I was like, “Oh no.”

(53m 37s) Tim Chaten:

Which is baffling, because Spritekit, it seems like the go-to 2D, kind of old-school style engine if you’re making, you know, like an NES-looking game, even though this game, of course, would not run on NES, there’s too much stuff going on, but that kind of general effect, yeah, lots of games would, I would think.

(53m 58s) James Swiney:

Yeah, I found it really good.

(53m 59s)

I think a lot of people shy away from it because it is Apple only.

(54m 2s)

So if you build something in SpriteKit,

(54m 4s)

you can only run it on Apple devices.

(54m 5s)

So whereas Unity, you can publish everywhere.

(54m 9s)

But anyway, as soon as I got the simulator,

(54m 12s)

I ran my games and to my surprise,

(54m 15s)

everything worked perfectly fine and SpriteKit runs wonderfully in it.

(54m 20s)

On the documentation I found it,

(54m 23s)

they still list SpriteKit as not.

(54m 28s)

recommended for the Vision Pro, and so the only thing I can get from that, or I can theorize anyway, is SpriteKit doesn’t support the focusing from the gaze tracking. So it obviously works because I’m playing around with my card games and some of the iPad ones, you can move them around

(54m 53s)

with the touching by looking, like using the mouse to…

(54m 58s)

be where you’re looking, but there’s no highlighting.

(55m 2s)

So in SpriteKit, I can’t tell if something is being focused on and highlight it.

(55m 7s)

Whereas in SwiftUI, your buttons, like when you look at the buttons in SwiftUI or hover over them in the simulator, you get this little highlight effect to show that that’s where you’re… Whereas in SpriteKit, there’s nothing.

(55m 18s)

So there’s no feedback, but that doesn’t affect me at all because I just care if they’re looking at the entire window.

(55m 26s)

Sorry.

(55m 28s)

But I can understand where if you were trying to build a UI in SpriteKit,

(55m 31s)

it just wouldn’t work because the user would have no idea what they’re interacting with, because there’d be kind of no feedback on what they’re currently looking at.

(55m 42s) Tim Chaten:

Your game, to like get into settings and to start a new game, I’d imagine there are buttons.

(55m 48s) James Swiney:

So, that’s all SwiftUI, yeah.

(55m 48s) Tim Chaten:

Are those in SwiftUI?

(55m 51s) James Swiney:

So, my latest, the way I’ve done my last few games is,

(55m 56s)

the game is all in SpriteKit, but all of the UI is in SwiftUI.

(56m 1s)

So, and that actually translates perfectly to the Vision Pro, just by luck.

(56m 7s)

Because, yeah, the UI that you need to interact with works fine with all of the

(56m 13s)

highlighting and focusing and that sort of thing, but the game still runs great, yeah.

How’d you get into game dev?

(56m 19s) Tim Chaten:

How did you get into game development?

(56m 21s)

How long have you been doing this?

(56m 22s)

Has this been something you’ve wanted to do for a long time now?

(56m 26s)

Just like, as a kid, is that something you’re playing games like,

(56m 32s) James Swiney:

The first game I ever modded was Grand Theft Auto 1 back in ’99, 2000.

(56m 41s)

So that was the top-down one.

(56m 44s)

And I found…

(56m 44s) Tim Chaten:

Yeah, for the original PlayStation, right?

(56m 46s) James Swiney:

Yeah, it was on PC. The DOS one was the one that I modded, or the Windows one.

(56m 52s)

And I found the scripting that they had for all the events and managed to make my own scripting and play around with it, that sort of thing.

(57m)

So from then on I’ve always been interested.

(57m 2s)

But I’ve kind of been on and off with it until the iPhone.

(57m 6s)

So I started developing iPhone in 2011.

(57m 11s)

So that would have been the iPhone 4, I think was around.

(57m 16s)

And yeah, I made a couple of really early games.

(57m 21s)

And then I worked on a card game.

(57m 26s)

This was, so my day job was still, I was like a systems administrator at a university.

(57m 32s)

And so I was doing this in my spare time.

(57m 34s)

And yeah, I just, I really loved doing the app development.

(57m 39s)

But I didn’t really care what it was.

(57m 41s)

I enjoyed making games in my spare time, but I didn’t really care what I was working on.

(57m 45s)

And I managed to land a professional job doing it.

(57m 49s)

And so my day job is still, I still have a professional day job where I do iOS development.

(57m 54s)

But all the game stuff I do after hours at night time and that sort of stuff.

(57m 59s)

And that’s my passion.

(58m 2s)

with making games for it.

(58m 4s)

I’ve been making so my biggest first game was Hectic Space 1 probably and then I made another shooter after that and then during covid my most ambitious game that I made was Aruna’s Adventure, which was the Zelda style game that I’ve made and that that’s when I found out I could burn out in lockdown working basically every night.

(58m 32s)

But I could, working on that.

(58m 34s)

By the time I got it out, I just didn’t want to touch anything anymore.

(58m 39s)

And I did burn out for a little while, but it’s all a learning lesson.

(58m 43s)

And now I know how to manage my time a bit better, so that I still do other things,

(58m 49s)

other than just the game stuff.

(58m 50s) Tim Chaten:

Arunas Venture, will that be on Vision OS, just not as optimized as this one?

(58m 56s) James Swiney:

So, that was the first one I played around with.

(59m)

And that’s one thing that got me excited about spatial computing.

(59m 4s)

And I did read a Mastodon comment from someone recently that was just…

(59m 9s)

Which was a very interesting thing to say.

(59m 11s)

It’s basically, all of your new apps that you support on the Vision Pro should support the spawning of Windows from that.

(59m 21s)

And so, the first thing I did is…

(59m 22s)

So, Rune’s Adventure is like you have an inventory that you can bring up

(59m 26s)

your items and all that sort of stuff.

(59m 28s)

So, the first thing I did is I went, "You can spawn your inventory,

(59m 31s)

but it appears as a new window next to you."

(59m 33s)

And then I’m like, “Oh, you can spawn your map and that appears as another window.”

(59m 36s)

So, you can have your map and your inventory and all these other windows always open while you play the game in the middle instead of having to open a menu or switch or that sort of thing.

(59m 45s) Tim Chaten:

That’s incredible.

(59m 46s)

I want to say that.

(59m 47s)

That’s going to be so much fun having like, I don’t know, this game with like three windows

(59m 52s) James Swiney:

Yeah, that’s it. Yeah. So, yeah, that’s probably something I’ll work on after.

(59m 58s)

I wanted to try it because I could definitely get… I don’t know the word for it, but where you try and focus on everything at once. And I definitely get into that.

(1h 7s) Tim Chaten:

Yes, you need to polish one, one, one game first, then move on.

(1h 12s) James Swiney:

That’s right. And one of the hardest parts of game development is, or any app development really,

(1h 18s)

is the final sort of 5-10%, which is like getting it.

(1h 22s)

Getting it out the door. I can get tons of cool effects happening in VoidX, but actually getting a shippable game for the Vision Pro is, yeah, so I wanted to focus just on that. And then after that,

(1h 34s)

yeah, I want to play around with, yeah, Aruna’s Adventure, sporting windows and that sort of stuff. I also have a card game that I’ve been maintaining for a long time, which it has multiplayer, so I’m interested in what I could do with that in Vision Pro as well.

(1h 52s) Tim Chaten:

That could be pretty interesting, like there’s share play, I don’t know, like FaceTime with that game in some way where you’re like looking at…

(1h 1m) James Swiney:

Yeah, so I already I have multiplayer in it, but yeah, shareplay would be really cool. I think I think being able to have

(1h 1m 6s)

Someone remotely also next to you and playing cards with them

(1h 1m 10s) Tim Chaten:

Yeah, like could you pull in their persona in some way? I don’t even know what the…

(1h 1m 10s) James Swiney:

Would be a really cool interaction

(1h 1m 14s)

Well, I think I think if you’re using the the shareplay

(1h 1m 18s)

Framework you get all of that for free. So you get the FaceTime call. You’re just doing something else in the FaceTime call So yeah, it should be two personas. And yeah, yeah, that’d be really neat

Art Assets?

(1h 1m 27s) Tim Chaten:

Do you partner with people that do the graphical work, or is that something you’ve figured out how to become an artist as well as a developer?

(1h 1m 37s) James Swiney:

I’m not an artist, no, and I tried for a while and it ends up just being super frustrating because I can’t get things to look how I want. The card games, I do all myself because that’s mainly just interface really, there’s not too much art that comes from it,

(1h 1m 38s) Tim Chaten:

Okay.

(1h 1m 58s) James Swiney:

it’s mainly just sort of design elements. But for Aruna’s Adventure, I ended up finding a really really good asset pack and then finding the artist.

(1h 2m 7s)

For it and getting getting more out from him and talking to him.

(1h 2m 11s)

And the same with Void X.

(1h 2m 12s)

I was avoid X.

(1h 2m 13s)

I actually found a poster of all the ships.

(1h 2m 16s)

So all of the ships in Void X there was a poster which was like a retro poster that was you could print out and I contacted the artist and said I’d love to use all these ships at a game.

(1h 2m 28s)

Can I can I get the rights to that?

(1h 2m 30s)

And he was first of all, he was shocked that someone would even ask he thought they just rip it off.

(1h 2m 33s) Tim Chaten:

[laughs]

(1h 2m 37s) James Swiney:

I worked it out with him and we worked out licensing so I could use all the ships in the game.

(1h 2m 42s)

And so yeah, I definitely now I like designing games and writing the code.

(1h 2m 49s)

I’m not an artist and I prefer to use other people’s wonderful

(1h 2m 53s) Tim Chaten:

Yes. Yeah. Yeah. Pixel art, especially, is a unique craft. It’s a whole different kind of art as well.

(1h 2m 59s) James Swiney:

Oh yeah, for sure. I do modifications to them and I’ve added animations to certain things in the games, but I always need to base it off someone else, what they’ve done, creating something from scratch in pixel art. I have no idea even what to, yeah, it always just looks wrong.

(1h 3m) Tim Chaten:

Pixel by pixel.

(1h 3m 21s)

So you’ll get the the assets and then to create an animation in sprite kit Do you like to say here’s like the first frame and the final frame and it kind of fills in the gaps or do you do?

(1h 3m 32s)

every every interstitial

(1h 3m 34s) James Swiney:

No, so it depends what the animation is.

(1h 3m 37s)

So, for example, like moving around,

(1h 3m 40s)

you’re just moving them around on the screen.

(1h 3m 43s)

But yeah, for all of the animations, like just in VoidX,

(1h 3m 48s)

there’s lots of simple animations.

(1h 3m 49s)

Like if you have a look, so in my previous games,

(1h 3m 52s)

I never had animations on the ships.

(1h 3m 54s)

They just kind of floated around.

(1h 3m 56s)

So in this one, I wanted it to be a bit more detailed.

(1h 3m 59s)

So every single ship has animations,

(1h 4m)

even if it’s just like a little sort of jet stream

(1h 4m 4s)

or something like that.

(1h 4m 5s)

And they’re all individual frames.

(1h 4m 6s)

So yeah, each ship has six, seven pictures.

(1h 4m 10s)

And then SpriteKit, you just tell it like every 0.1 of a second, change the frame and it runs the animation.

(1h 4m 16s)

Yeah, it’s time consuming creating the animations,

(1h 4m 21s)

but or creating the individual frames anyway.

(1h 4m 23s)

But for Aruna’s Adventure, thankfully,

(1h 4m 26s)

all of the enemies had animations already done.

(1h 4m 30s) Tim Chaten:

Oh, nice. For the fully immersive mode of VoidX, did you play around with making the ship’s like three-dimensional coming out a bit of the back of the screen?

3D Ships?

(1h 4m 46s) James Swiney:

Yeah, so there’s a couple of things I played with.

(1h 4m 47s)

The first thing I tried, back to not being in the immersive screen,

(1h 4m 53s)

I tried actually having a background, like basically two scenes,

(1h 4m 59s)

one was just running the background and one was just running the ships,

(1h 5m 3s)

and then I tried to space them, I guess, sort of 20 centimetres apart,

(1h 5m 8s)

so it kind of would give the effect of the background moving independently of them.

(1h 5m 14s)

It didn’t really work.

(1h 5m 16s)

In the simulator anyway, I couldn’t really see the effect, and I guess that’s the other problem using the simulator.

(1h 5m 23s) Tim Chaten:

• Right, it’s not a three-dimensional.

(1h 5m 23s) James Swiney:

It’s hard to like… Exactly.

(1h 5m 24s) Tim Chaten:

You need it developing on a 3D screen in some way.

(1h 5m 28s)

I don’t know if even Xcode would support that though.

(1h 5m 29s) James Swiney:

Yeah, yeah. So all I was doing was moving the camera around in the simulator, trying to see on an angle how it would look.

(1h 5m 36s)

I ended up deciding it wasn’t really working that well.

(1h 5m 41s)

I did think about at some stages trying to build a voxel ship or something as well.

(1h 5m 47s)

But then again, I went back to basics.

(1h 5m 50s)

The city is really interesting, how I’m doing that.

(1h 5m 54s)

I’m using Apple’s sample idea.

(1h 5m 57s)

So in one of Apple’s sample codes, they have a solar system.

(1h 6m 1s)

And so you launch the immersive scene and you’re in a solar system.

(1h 6m 5s)

And everything is black around you because you’re in space, except for a sun.

(1h 6m 10s)

There’s like a sunlight shining from one side.

(1h 6m 13s)

and I looked how they were doing it and…

(1h 6m 16s)

It was really interesting. They basically put the user inside a gigantic sphere and invert the texture on it. So, basically you get a massive sphere and put a texture on the outside, but then you sort of invert it so it’s facing in.

(1h 6m 30s)

And so, yeah, it’s like a… I can’t remember how big it is.

(1h 6m 33s)

It’s like a thousand meter sphere or something.

(1h 6m 36s)

So I tried that in VoidX. Just made a thousand meter black sphere beer that you’re sitting in, and then made a huge image that’s

(1h 6m 46s)

the cityscape, so that it’s long enough to wrap around you. The effect in the game looks surprisingly effective, which is I assume why Apple are doing it as well. It’s a neat trick

(1h 7m)

to try and remove the background, put you in a big circle.

(1h 7m 6s) Tim Chaten:

Yeah, mimicking the sun in VR is actually getting to the point where it actually kind of works.

HDR

(1h 7m 11s) Tim Chaten:

Like I know in PSVR 2, Call of the Mountain, you kind of like kind of blind yourself a little bit because the HDR is, it gets pretty bright and it’s like, Oh, I’m looking at the sun.

(1h 7m 21s)

Grand Turismo is the same.

(1h 7m 23s)

You’ve probably experienced this where the sun reflects.

(1h 7m 28s) James Swiney:

Gran Turismo Sun is incredible in PSVR 2.

(1h 7m 31s)

Gran Turismo Sun, I almost feel like it’s warm when it’s hitting the dash of the car.

(1h 7m 34s) Tim Chaten:

Yeah. I’ve only done the snow levels through the PSPortal. I want to try them in VR to see if there’s like sun reflecting on the snow in cool ways. I imagine there will be.

(1h 7m 35s) James Swiney:

Like, yeah, it’s really cool.

(1h 7m 48s)

oh yeah yeah that one

(1h 7m 51s) Tim Chaten:

But anyways, anything else on Vision Pro or Vision OS?

Anything else?

(1h 7m 57s) James Swiney:

No, I think we’ve pretty much covered everything that I’ve had experience with.

(1h 8m 6s)

I’m just really excited to get something out on there.

(1h 8m 10s)

It’s exciting to be there for day one on a new device and especially to be making something for it as well.

(1h 8m 16s) Tim Chaten:

No, it’s got to be pretty cool as you’ve been there for day one on Apple Watch with the company you’re at.

(1h 8m 22s)

Is this anything were you developing back for iPad or iPhone era?

(1h 8m 29s) James Swiney:

So, I remember the big change, which I actually,

(1h 8m 34s)

well, I was only learning at the time.

(1h 8m 37s)

I was pretty new to it when the iPhone 5 came out and the screen size changed and that was a nightmare because everything was just hard-coded to that one screen.

(1h 8m 46s)

And so, I remember that being a real struggle.

(1h 8m 49s)

But yeah, I can’t, I think, yeah, this is,

(1h 8m 54s)

yeah, the Apple Watch was sort of the first big thing thing as a new sort of Apple computing.

(1h 8m 59s)

device. I have actually released, as soon as the M1 Macs came out, so I optimised. I’ve got a couple of my games running on the Mac as well. So, Aruna’s Adventure has a native Mac version.

(1h 9m 16s)

So, I think this is going to be one of the first ones where I’m actually there,

(1h 9m 19s)

like, yeah, doing my own stuff for it.

(1h 9m 22s) Tim Chaten:

Yeah, very cool.

(1h 9m 23s)

It’s always exciting.

(1h 9m 24s)

A brand new app store is just a fun place to explore and just look around like, “Oh, what do we have here?”

(1h 9m 29s) James Swiney:

Yeah, yeah, for sure.

(1h 9m 30s) Tim Chaten:

Yeah, yeah, I have very fond memories of the iPad app store,

(1h 9m 33s)

that initial launch library,

(1h 9m 36s)

and I miss many of those titles that are no longer with us, like Scrabble.

(1h 9m 44s) James Swiney:

Oh, yeah. I miss the skeuomorphic designs.

(1h 9m 49s)

I often go back and look at any iOS 6 apps and stuff.

(1h 9m 54s) Tim Chaten:

Yeah, it was a different era.

(1h 9m 54s) James Swiney:

And I just love that style.

(1h 9m 56s)

I recently did a talk at a meet up about Game Center and the history of Game Center.

(1h 10m 4s)

And the Game Center with the wood grain and the felt on the air.

(1h 10m 10s)

That was awesome.

(1h 10m 10s) Tim Chaten:

Yeah, yeah, Game Center is something I’ve never truly wrapped my head around like it’s

(1h 10m 10s) James Swiney:

I love that.

(1h 10m 11s)

Yeah.

(1h 10m 17s) Tim Chaten:

Like you get achievements and stuff, but it’s like I don’t know it’s just it’s there But it’s it’s no PS. You know plus or profile

(1h 10m 28s) James Swiney:

Yeah, so it was a big deal back in the iOS 6 days because it had its dedicated app.

(1h 10m 34s) Tim Chaten:

Yeah, there’s no app anymore, right?

(1h 10m 34s) James Swiney:

It was kind of like the Xbox Live for iPhone games because it was where your profile was,

(1h 10m 36s) Tim Chaten:

Yeah.

(1h 10m 42s) James Swiney:

your achievements, your friends list, leaderboards, all that sort of stuff.

(1h 10m 46s)

And then they got rid of the app and then it kind of fizzled for a while.

(1h 10m 53s)

But with iOS 14, I think it was, there’s been lots of great changes for it.

(1h 10m 58s)

The achievements look amazing now and so do the leaderboards.

(1h 11m 2s) Tim Chaten:

How do you find all of your achievements in one place? Does that exist?

(1h 11m 7s) James Swiney:

Yeah, so not really.

(1h 11m 9s)

They have what’s called an access point that you can put in your app.

(1h 11m 14s)

So I put it in my apps and it puts a little button up the top corner and you can open that and see achievements and leaderboards and that sort of stuff.

(1h 11m 21s)

And they seem to be leaning quite heavily into the widget.

(1h 11m 24s)

The Game Center widget lets you have that sort of stuff on your home screen.

(1h 11m 28s)

I would love them to bring back the Game Center app because there’s plenty that you can look at.

(1h 11m 32s) Tim Chaten:

Yeah, that seems like an essential thing. Yeah.

(1h 11m 34s) James Swiney:

I use Game Center for turn-based multiplayer in my card game.

(1h 11m 40s)

They make that super easy. They handle notifications, they handle matchmaking.

(1h 11m 46s)

All that sort of stuff is really cool. There’s lots of cool stuff in Game Center.

(1h 11m 50s)

It’s just kind of hidden from most people and most people don’t know what it is yet.

(1h 11m 54s) Tim Chaten:

• Yeah, like they should have an app.

(1h 11m 57s)

From that app, you can find all the games that have multiplayer.

(1h 11m 59s)

You can invite people to play a game from that app.

(1h 12m 2s)

That seems like no-brainer.

(1h 12m 4s) James Swiney:

So you’ve just made me want to try something, I’m going to log into the simulator as my Apple account so that I can pull up Game Center.

(1h 12m 14s)

I’ve never seen the Game Center interface in the Vision Pro, so I’m curious to see what that looks like.

(1h 12m 17s) Tim Chaten:

Yeah, see what that’s like.

(1h 12m 20s)

Yeah, absolutely.

(1h 12m 21s)

Well, James, thank you so much for your time today.

(1h 12m 24s)

This has been an awesome chat and I’m very excited to try this game, both of those games,

(1h 12m 28s)

out on my Apple Vision Pro next year at some point.

More info?

(1h 12m 31s) Tim Chaten:

Where can people find your different games and what you’re up to?

(1h 12m 37s) James Swiney:

Oh yeah, thanks so much for having me. It’s been great chatting with you.

(1h 12m 41s)

You can find me on, it’s just @jamessweeney, my name, on Twitter and Mastodon and jamessweeney.com

(1h 12m 52s)

or you can search my name on the App Store and that’s where all my apps are.

(1h 12m 56s) Tim Chaten:

Excellent. Thank you so much. Really appreciate it

(1h 12m 59s) James Swiney:

No worries, thanks so much.

---

Made with Transcriptionist