Vision Pros

Transcript of the Interview

(2m 22s) Tim Chaten:

It’s exciting times in Apple Vision Pro land.

(2m 25s)

We keep itching closer and closer to this inevitable release sometime early next year,

(2m 30s)

according to Apple, as they reiterated recently.

(2m 33s) Clarko:

Early, who’s to say?

(2m 34s) Tim Chaten:

Early, yes.

(2m 37s)

Fingers crossed for February or even January.

(2m 41s)

What a world.

(2m 42s) Clarko:

In my internal clock, I’m like putting it on April fool’s cause I’m like, that’s still early.

(2m 46s)

And that’s like a historically relevant date in Apple history.

(2m 49s)

So let’s shoot for that.

(2m 50s) Tim Chaten:

Yeah, yeah, why not?

(2m 52s)

So, first let’s just dive into your background as a developer and kind of Apple user in general before we dive into everything else, which there’s a lot to chat about.

(3m 2s) Clarko:

Sure, yeah, so I’ve been a Mac user and an interface designer for 20 some years now.

(3m 9s)

Before that, before that I was a Windows kid. I was always interested in other operating systems,

(3m 15s)

so to the extent that I could get my hands on them because I grew up on MS-DOS.

(3m 18s) Tim Chaten:

It’s always the fun thing as a kid, like I had Windows kid as well, it’s like you don’t have any money, you can’t really, you can’t like ask, but it’s like good luck, yeah, right

(3m 28s) Clarko:

Yeah, we’re building our own computers.

(3m 29s)

But we, like I grew up on MS-DOS and we had acorn machines at school and acorn RISC machines are what became the ARM alliance.

(3m 40s)

That’s like a weird historical glitch, but you know, I had buddies with an Amiga and my auntie had an old Mac.

(3m 47s)

And so like Macs were just like one of many computers I didn’t have.

(3m 51s)

And then, uh, I did an internship in an ad agency in 10th grade and they were all using Macs.

(3m 56s)

And when I started getting into web design.

(3m 58s)

And web development, that whole crowd was using Macs and I started studying interactive multimedia and that was all Macs.

(4m 4s)

And so I just got to using them.

(4m 6s)

And OS 9 was fine.

(4m 8s)

It was crashy.

(4m 10s)

And it had its interesting things about it, shall we say?

(4m 15s) Tim Chaten:

Sure, I only used it in retrospect after it was retired.

(4m 16s) Clarko:

I mean, coming from Windows.

(4m 18s)

Yeah.

(4m 19s) Tim Chaten:

Yeah.

(4m 20s)

Mm-hmm.

(4m 20s) Clarko:

But this was around the time everyone was starting to upgrade to OS 10.

(4m 21s) Tim Chaten:

Yeah.

(4m 22s)

Yeah.

(4m 23s)

Yeah.

(4m 24s) Clarko:

And they were all a little nervous.

(4m 26s)

I decided particularly that it was more.

(4m 28s)

Able than I was nine, but also it was like technically impressive and it looked incredible.

(4m 33s)

And so I ended up buying an emac with my education discount, which is the cheapest thing you could possibly buy.

(4m 38s)

And I was just kind of shocked by how nice it was that it was like software.

(4m 46s)

It’s like software that was made with craft and care and that whole just it just works ethos of the Apple ecosystem.

(4m 52s)

And I wanted to do that.

(4m 54s)

Like I wanted to make nice tools for people who like nice tools.

(4m 57s) Tim Chaten:

Yeah. Cool. Yeah, that was the era of, uh, you could shift click on the minimize button to delete the slow genie animation. I think that’s been yanked out at this point.

(4m 58s) Clarko:

So here I am.

(5m)

I’ve been lucky to work on a lot of stuff.

(5m 7s) Tim Chaten:

Yeah. Yeah. But yeah, I, yeah, I got into the era of Macs in 2006. So a little bit after, uh,

(5m 8s) Clarko:

I haven’t tried that in a long time. I’d be surprised.

(5m 15s)

Mmm.

(5m 16s) Tim Chaten:

the transition to, to Mac OS X, but that was still the fun kind of era where everything was very,

(5m 23s)

playful. Yeah.

Interface Designer

(5m 27s) Tim Chaten:

So interface designing, what kind of projects have you taken on and what kind of different

(5m 37s) Clarko:

I’ve been pretty lucky because I moved out to the United States when I was in my mid twenties.

(5m 44s)

I grew up in Australia, despite the accent, and fell into startup land.

(5m 51s)

And I’ve been able to work on MLB at that, and I worked at Square and on Fitbit.

(5m 56s)

And then I got to work on whole operating systems for smart appliances.

(6m)

I worked on the June oven and Weber grills.

(6m 3s)

And nowadays I do a mix of freelance prototyping for different clients and.

(6m 7s)

Building my own apps for Apple platforms.

(6m 9s)

So I’ve always been that designer who was very much more on the technical end,

(6m 13s)

who was contributing to the code base.

(6m 15s)

And nowadays I get to kind of sit right in the middle with like prototyping in the middle of design and development.

(6m 17s) Tim Chaten:

Yeah Very cool and designing interfaces how much into art and drawing and creating the art part of the design do you have to Have in your tool set

(6m 35s) Clarko:

It’s funny, like I said, I’m on the technical end of that design spectrum, and when I first started working in Silicon Valley, I was working exclusively with designers who are on the art end of the spectrum.

(6m 47s)

And so I always envied them that they were so much better at that part of the job.

(6m 52s)

And this was during those days when everything was fully skeuomorphic, and they’re like creating these incredible textures and doing the hand stitching on leather buttons.

(7m 1s)

And I was like, I can’t do that.

(7m 3s)

But what I can do is like.

(7m 5s)

Really great information architecture and like pick out these weird little bugs that are confusing to people and I can I’m very good at documentation. So it’s like it’s a totally different part of the job, but you know, I have drawn my own icons my whole career, just not to the level of fidelity of like your your Robert Anderson’s and your Louie Mantia’s and your bad else’s.

(7m 23s) Tim Chaten:

Yeah, yeah, the iOS 6 era was something else like I kind of missed that stuff. Yeah, there was a an iPad app that came out recently by the Halide guys that remind me of that era that does the It pulls up to your iPad as a HDMI monitor I’m gonna pull up the name here real briefly cuz I forget off Orion yeah, and and it’s got this Orion operating system kind of thing to it. It’s got a just a very playful phone app

(7m 25s) Clarko:

It was.

(7m 29s)

[Chuckle]

(7m 45s)

It’s called Orion, I think, yeah, and it’s got like a CRT emulation mode.

(7m 50s)

Yeah, especially for like something that does something relatively simple,

(7m 53s) Tim Chaten:

Yeah Yes Yeah Yeah, the the the great design thing. I love and you like drag your finger to unbox the app kind of thing. It’s so Skeuomorphic in a delightful way that I miss Peace.

(7m 58s) Clarko:

like letting yourself stand out that way is so cool.

(8m 8s)

Yeah, and Sebastian, who heads up design over there, like he is of that same era.

(8m 18s)

Like I know that he misses those days.

(8m 19s)

So he’s like throws it in as a little Easter egg whenever he can.

(8m 20s) Tim Chaten:

Yeah. Yeah.

Current Hardware Setup and Gaming

(8m 23s) Tim Chaten:

So what’s your current hardware set up these days?

(8m 27s)

What Mac, you rock an iPad.

(8m 29s) Clarko:

I’m all over the place.

(8m 33s)

So at my desk I have an M1 iMac, which is purely because that’s suited to being at a desk.

(8m 37s) Tim Chaten:

Okay. It’s a perfect desk computer.

(8m 44s) Clarko:

And I have an M1 MacBook Pro, the one with the touch bar.

(8m 49s)

I bought it because it had the touch bar.

(8m 50s)

I was like, "I’ve never used it.

(8m 52s)

I’ve got to use it before it disappears."

(8m 52s) Tim Chaten:

Sure, yeah.

(8m 54s) Clarko:

And a few months later, they introduced, you know, the new model for M1 Mac.

(8m 54s) Tim Chaten:

Naturally, yeah.

(8m 59s) Clarko:

And so I missed out on that, but I feel like I’m touching history.

(9m 5s)

Um, and iPad pro is kind of my day to day workhorse.

(9m 12s)

I use the iPad pro in the magic keyboard so much because I use it for Swift prototyping, because it’s just, you don’t have to hit compile.

(9m 20s)

You just type some text and then you can touch it on the screen immediately.

(9m 23s)

It’s so satisfying for prototyping and with pencil, it’s so satisfying for drawing.

(9m 29s)

I like sit on the couch and do some design work tool and then just, you know, a phone.

(9m 34s)

[inaudible]

(9m 35s) Tim Chaten:

Excellent. Yeah, we will need to have a different conversation all about iPad usage someday for iPad Pros, because that sounds awesome.

(9m 39s) Clarko:

[laughs]

(9m 41s) Tim Chaten:

And do you own any headsets of any kind? Currently, they’re mostly just for gaming, but I’m curious.

(9m 52s) Clarko:

I do. I got on to the Quest 2 in a big way during those early months of the COVID pandemic.

(10m)

And the price point obviously makes it really accessible. It’s, you know, less than a tenth of the cost of a Vision Pro at retail, even though they are upping the price for the next generation, which they just announced.

(10m 7s) Tim Chaten:

Yes. Yes. Yeah, $500, which is still very reasonable. I think $550 was what the PSVR

(10m 22s) Clarko:

So, you know, once I explored the catalog and played all the games that I felt like I wanted to play, it didn’t have any everyday utility for me. Like I tried the web browsers and I tried some of those other tools and it’s like the screens and the lens artifacting,

(10m 47s)

It’s not pleasant to use for text or normal 2D video.

(10m 51s) Tim Chaten:

Right.

(10m 52s) Clarko:

Navigating with the WANs or the controllers, it feels like you’re using a laser pointer.

(10m 57s)

It’s very wobbly, and so I’d love to see a port of some of those titles for other platforms.

(11m 3s)

I have been looking at the PSVR2 because one of my favorite games called Moss,

(11m 5s) Tim Chaten:

Mm-hmm Delightful in that platform. Yes, it is so good Yeah, right Yeah, the the future of Apple Vision Pro with gaming is just gonna be a curious one cuz The hand tracking is everything right now, and you have you can use a 2d flat game controller

(11m 7s) Clarko:

which is a little adventure where you’re a mouse.

(11m 10s)

Oh, so cool.

(11m 12s)

So I’d love to see ports of all of those titles, but I’m not counting on it.

(11m 17s)

You know, Apple’s reputation with game devs has always been kind of so-so.

(11m 33s) Tim Chaten:

Which, you know, great.

(11m 35s)

For 2D games, flat games, but, uh, yeah, if you start to it’s it’s ground up VR gaming.

(11m 41s)

And I think that’s probably where I will continue to play my VR games because it has more horsepower.

(11m 48s)

I think GPU is and all that stuff being a big, huge box dedicated to gaming for several vision pro on an M2 chip.

(11m 55s)

And that could change the future.

(11m 58s)

But until they figure out the controllers, because I do feel like haptics in your hands and actual buttons.

(12m 5s)

Are useful in video games.

(12m 7s)

I mean, as we’ll talk about, they’re not going to be totally useful if you’re trying to use it as a computer, but for gaming, totally useful.

(12m 8s) Clarko:

Absolutely.

(12m 9s)

Yeah, and I do wonder about this whole, like, metal migration kit that they announced this year and some of the carrots they’re dangling for game devs, because even if you can’t use the spatial controllers from a VR headset, it would be nice to just bring in some, like, classic flat screen games, pick up your PS4.

(12m 38s)

PS5 controller with its incredible haptics and play in the headset on like a far bigger screen than you could have in the home otherwise.

(12m 40s) Tim Chaten:

Mm-hmm Yeah, and there are some solid VR games that use traditional controllers PSVR one had a ton of those because the move era wasn’t great with that So there’s a lot of games that did just use the regular ones and you know super hypercube was great I mean Tetris effects connected uses the PSR to controllers, but it’s also very enjoyable

(13m 6s)

with just a regular controller. I mean some games just work.

(13m 10s)

very well in that way and you’re still immersed in that world where you can kind of zone out in Tetris and it’s great. Yes, that is just, it’s, you know, Apple says there’s this whole meditation thing. That’s my meditation in VR. I’ll just load it up. It’s just, it’s delightful. But yeah, yeah, the Quest 2, I imagine, yeah, web browser would just be miserable there.

(13m 17s) Clarko:

Yeah, Tetris Effect kind of just works on every platform, it’s great.

(13m 38s)

Yeah.

(13m 38s) Tim Chaten:

Yeah, uh, you mentioned–

(13m 40s)

It is a PlayStation 2D video, which I find quite enjoyable in PSVR 2. It’s 2K per eye, so I know Apple Vision Pro will be double that and just blow this out of the water, but, um, it is a much different experience holding, like, an iPad a couple inches, you know, you know, foot from your face versus wearing PlayStation VR 2 and watching, you know, Foundation on that. It is, it’s, it’s like I got a projector in my house, like, a movie theater in my house. It’s, it’s, it’s great.

(14m 8s)

So I am looking forward to uploading.

(14m 10s)

So as far as Apple Vision Pro, when you saw this announced, you’ve had this experience with Oculus or Quest 2 prior.

Apple Vision Pro

(14m 31s) Clarko:

I still call it Oculus.

(14m 35s) Tim Chaten:

What was your first reaction?

(14m 37s)

this event I was like oh I’m…

(14m 40s)

I don’t really care what Apple’s doing and then I saw their reveals like oh this is for everything else. This is a computer.

(14m 46s)

What was your kind of edition?

(14m 47s) Clarko:

Yeah, I was very excited because, yeah, I’m I’m not a serious gamer.

(14m 53s)

I have a Switch and a PS5 was a very recent addition to the household.

(14m 56s) Tim Chaten:

Yeah.

(14m 57s)

Yeah.

(14m 57s) Clarko:

But the rumor mill had been reporting on hardware all year and they’d kind of done it to death.

(14m 58s) Tim Chaten:

Yeah.

(14m 59s)

Yeah.

(15m)

Yeah.

(15m 1s)

Yeah.

(15m 2s) Clarko:

And so the OS reveal was like this huge surprise.

(15m 6s)

It was just, you know, wonderful to see thinking about things that you you didn’t know what you’d get from from the hardware rumors because pass through cameras on by default.

(15m 17s)

Is really novel.

(15m 19s)

The windowed interface, which is clearly designed for multitasking,

(15m 23s)

is, you know, really nothing like what we expected from other headsets.

(15m 26s) Tim Chaten:

Right?

(15m 27s) Clarko:

And this whole bit about driving the interface with your eyes,

(15m 30s)

with your hands in your lap or standing.

(15m 33s)

That really struck me as way better ergonomically than hunched over a desk.

(15m 38s)

And for someone who, you know, I’m watching relatives get older.

(15m 42s)

And, you know, as they my grandfather who who passed a couple of years ago

(15m 47s)

watching him use a phone was really something.

(15m 50s)

It was like a real empathy exercise because he just didn’t have the dexterity of a younger person.

(15m 56s)

And so I just feel like using your eyes and then just kind of like pinching your fingers will extend the useful life of technology for so many people in so many ways.

(16m 7s)

So I was super excited a lot by that for some weird reason.

(16m 11s)

But my my mind was racing.

(16m 13s) Tim Chaten:

• Yeah.

(16m 13s) Clarko:

I just came out of that keynote with a ton of ideas.

(16m 13s) Tim Chaten:

Yeah.

(16m 17s)

No, I’m super excited to use it.

(16m 20s)

We have a newborn in the house, a one-year-old now.

(16m 24s)

And she does a lot of sleeping at night on me before we put her down to the crib.

(16m 28s)

And it’s nearly impossible to use an iPad as a computer and that set up with her on my chest and stuff,

(16m 34s)

like it as a laptop or whatever.

(16m 36s)

And holding the iPad’s okay,

(16m 39s)

but I gotta make sure I’m not getting too tired and no dropping.

(16m 43s)

Yeah. Yes.

(17m 4s) Clarko:

Yeah, I have a two-year-old dog, which is a totally different ballgame, but he’s very demanding and he doesn’t like it when I’m using an iPad in front of him.

(17m 10s) Tim Chaten:

[laughs]

(17m 12s) Clarko:

And so I’m just very curious how he’ll react to headset daddy and just, I’ll still pet you but I’m not looking at you, I’m sorry.

(17m 13s) Tim Chaten:

[laughs]

(17m 15s)

[laughs]

(17m 19s)

Yeah, you could actually be petting the dog.

(17m 27s) Clarko:

Yeah, I mean, so long as you’re not petting in a way that it looks like you’re pinching, just keep one hand free.

(17m 30s) Tim Chaten:

[laughs]

(17m 31s)

Right.

(17m 32s)

Yeah.

(17m 34s)

So you were the very first guest I’ve had on that has been to the Apple Vision Pro Labs.

Vision Pro Labs

(17m 42s) Tim Chaten:

I was gonna say first person that’s used it,

(17m 45s)

but that’s not true.

(17m 45s)

Chance Miller did the press experience.

(17m 49s)

So he had 30 minutes.

(17m 50s)

You had a good five to eight hours from what I’ve heard online of people going to these.

(17m 55s)

It’s a very full day, is what I’ve heard, yeah.

(18m)

Before we get to all of what you can say,

(18m 3s)

which is probably not a ton or anything from that experience, spatial computing in general,

20 Years from Now

(18m 10s) Tim Chaten:

you know, 20 years from now as technology improves,

(18m 12s)

is this gonna, in your mind, replace for most people,

(18m 17s)

desktop computing, phones?

(18m 18s)

Like what’s your aspirations for spatial computing and Vision OS in general?

(18m 25s) Clarko:

I mean 20 years is a really long time. I think…

(18m 28s) Tim Chaten:

I figure, you know, two decades, that’s…

(18m 29s) Clarko:

Yeah, oh yeah. But if you had asked me 20 years ago if I thought cell phones were the future,

(18m 30s) Tim Chaten:

seems like sufficient time for technology to miniaturize, right?

(18m 38s) Clarko:

and I had a Sony Ericsson T68i in my hand, I would have been skeptical.

(18m 44s)

But I don’t know, this feels like a huge leap in computing interfaces, and especially in like work computing interfaces, and there are a lot of kinds of

(18m 55s)

work. Because I guess like starting next year, there’ll be enthusiasts like us who will tolerate the size and the weight and the battery life and we’ll remote desktop into our Mac, and we’ll run our iPads in compatibility mode. And like,

(19m 8s)

this all feels very much like the early days of OS 10, where we’re all just kind of like, duct taping together our old computing lives onto our new computing life. And we’ll tolerate the sharp edges, because that’s what that’s what enthusiasts do. But then after us, there are industry

(19m 11s) Tim Chaten:

Right Yes

(19m 25s) Clarko:

full of people who will benefit from spatial computing in real and interesting ways that they’re using 3d software all day today, but they do it on a 2d screen. And they can only talk about their work in 2d terms, even though it’s 3d work, like interior designers, industrial designers,

(19m 44s)

architects, jewelers, mechanical engineers, and it’s like weird,

(19m 47s)

I’ve worked with a lot of these people in my career. And I, you know, reviewing their work product is actually kind of

(19m 54s) Tim Chaten:

Yeah, I remember in high school or even middle school, we used CAD software on these Windows computers and we’re working on these 3D projects. I can only imagine that space.

(20m 7s) Clarko:

Yeah, I think they’ll be on board quicker than most, although it depends a lot on the tools.

(20m 11s) Tim Chaten:

Right, yeah, the software. I wonder, and I wonder, like, Finale never, a music notation software, Finale never made the jump to iPad or iOS or any of that stuff.

(20m 23s)

We have new companies that have taken that mantle, and I wonder, in the CAD world, will the old hats make that migration, or will it take a new company?

(20m 38s) Clarko:

Yeah, I’m really, even if the old companies do just output compatible files, I think that will be a big change in workflow because if, if they can export USDZ and then you can throw that just email someone a USDZ file and let them review it because so much of that job is going into meetings where you look at like side renders of a 3d object and you’re like,

(21m 3s)

okay, yeah, that that’s okay.

(21m 4s)

And then you give feedback and then it goes for another round.

(21m 6s)

And then once it reaches.

(21m 8s)

A point of maturity in the design process, they start to do physical prototypes.

(21m 13s)

And those are expensive and time-consuming.

(21m 16s)

And even if it’s just cardboard packaging, which like I’ve worked on a ton of cardboard packaging in my career.

(21m 21s)

And it’s like, that’s a lot of work to go to the art store and buy some artboard and get out your bone folder and like produce a fake box.

(21m 29s) Tim Chaten:

Yeah.

(21m 29s) Clarko:

Just so an executive can look at a fake box.

(21m 32s)

And if you just do that at one-to-one scale in a headset, it would be such a time-saver.

(21m 38s) Tim Chaten:

Yeah, and you could have the object that you’re going to be packaging with it in real life kind of like next to it and like, yeah, that could be fun and useful.

(21m 48s) Clarko:

Yeah, like if it’s on a table in front of you, yeah, and, you know, that’s, that’s one niche and we’re a different niche and, but putting my startup hat on like spatial computers could become like standard issue gear in new offices.

(21m 50s) Tim Chaten:

Yeah, right.

(22m 6s)

Mm-hmm. No.

(22m 6s) Clarko:

I don’t think anyone’s about to like retrofit their old office because it’s a lot of expense,

(22m 11s)

but office space and office furniture are really expensive, like horribly expensive ’cause the whole.

(22m 18s)

• Yeah.

(22m 18s)

• So I was like,

(22m 19s)

“What if you just didn’t buy those?”

(22m 20s)

What if you had like a comfy chair and a cubby where you can put your stuff and store your headset when you’re not using it?

(22m 23s)

It’s just the,

(22m 23s)

this is a horrible thing to think of,

(22m 24s)

but like they’ve been pushing for less space per employee for so long in every industry.

(22m 26s)

And if everybody didn’t need a desk,

(22m 27s)

they would be like,

(22m 28s)

“Oh, I’m gonna have to buy a desk.”

(22m 28s)

• Yeah.

(22m 29s)

• And it’s like,

(22m 30s)

“Oh, I’m gonna have to buy a desk.”

(22m 31s)

• Yeah.

(22m 32s)

• And it’s like,

(22m 33s)

“Oh, I’m gonna have to buy a desk.”

(22m 33s)

• Yeah.

(22m 34s)

• And it’s like,

(22m 35s)

“Oh, I’m gonna have to buy a desk.”

(22m 36s)

• Yeah.

(22m 37s)

• And it’s like,

(22m 37s)

“Oh, I’m gonna have to buy a desk.”

(22m 38s)

• Yeah.

(22m 39s)

• And it’s like,

(22m 40s)

“Oh, I’m gonna have to buy a desk.”

(22m 41s)

• Yeah.

(22m 42s)

• And it’s like,

(22m 42s)

“Oh, I’m gonna have to buy a desk.”

(22m 43s) Tim Chaten:

Chuckles Yeah.

(22m 43s) Clarko:

• Yeah.

(22m 44s)

• And it’s like,

(22m 45s)

“Oh, I’m gonna have to buy a desk.”

(22m 46s)

• Yeah.

(22m 48s)

• And they would like jump up and down about the square footage that they’re no longer paying for.

(22m 52s)

And if working from home and working from the road were just as productive as working from the office,

(22m 59s)

also a huge gain.

(23m)

So I don’t know,

(23m 1s)

this adoption curve in industry seems like it could be really weird and that’s not Apple’s usual thing.

(23m 8s)

So this is a really expensive consumer product,

(23m 10s)

but like the business spending analysis is,

(23m 15s)

I don’t know how to make that.

(23m 17s) Tim Chaten:

Yeah, and designing office space around headsets is a weird one, because you benefit from wide open spaces and spatial computing, not these little tiny cubbies they’ve been carrying people in.

(23m 27s)

And another thing to consider is if you have a huge wide open space, like a big table everyone’s kind of working around, you have all the people using the same space virtually in a way that’s that’s just kind of interesting.

(23m 44s)

Like everyone’s looking at different things.

(23m 47s)

Save virtual space.

(23m 49s)

Yeah, and then you’ll be able to enter total VR mode if you do want to isolate from the other people around you.

(23m 56s)

Maybe throw on AirPods Pro with noise cancellation to try to really, like, I’m not paying attention to anything else but this right now.

(24m 2s)

And so you kind of get benefits of being in your own office, sorta, without needing that.

(24m 4s) Clarko:

Yeah, I mean, yeah, I mean, in the in the open floor plan offices that became trendy,

(24m 9s) Tim Chaten:

Yeah.

(24m 10s)

Yeah, everyone hated.

(24m 14s) Clarko:

you know, 10, 15 years ago. Yep. And I went straight to noise cancelling headphones,

(24m 20s)

unless I was in a mode of work where I wanted to overhear stuff, which is like, what is interesting about office work that it’s, you know, that it is inherently social, if you’re not trying to get

(24m 31s) Tim Chaten:

Yes. You can waste eight hours very easily with the social stuff if you want to.

(24m 37s) Clarko:

[laughs]

(24m 38s) Tim Chaten:

It’s like I had a very productive day of working with my co-workers, but oh, my projects didn’t really move forward today, did they?

(24m 45s)

No. Yes, yes.

(24m 45s) Clarko:

It’s networking, it’s meetings, it’s, you know.

Pleading the 5th

(24m 48s) Tim Chaten:

I’ll reference John Gerber’s The Talk Show. He called it Pleading the Fifth with Michael Simmons of Flexibits.

(24m 58s)

He also went to the lab, um, as well.

(25m 1s)

Um, so basically I believe all the only things you’ll be able to really chat about are things that are publicly known, like kind of like from the press demo.

(25m 12s)

So you know, uh, so whatever you feel comfortable sharing, uh, I will have questions and some of them we will just have to skip right by depending on, on how we, we feel it goes here.

(25m 24s) Clarko:

Yeah, honestly, being that I’m like putting on my designer hat, it’s there isn’t that much about Vision OS that isn’t public, because that keynote was a big software demo. They’re showing it off,

(25m 37s)

they’re trying to get people hyped for it. So they put it all out there. And then they have the interface guidelines that they publish. And they have all these videos from dub dub that are part of those interface guidelines. And they really go in depth about the design of the user interface. And so there’s a ton of things we don’t know about the hardware, there’s tons of

(25m 54s)

things we don’t know about the OS from a technical level. But like the UI is out there. We there we have plenty to talk about. Yeah, but WWDR is very clear that we shouldn’t share details about the labs because it’s not a press demo. And I’m not a journalist. So I will catch things in terms of everything I’ve said publicly. But full disclosure, I’m friends with a lot of Apple people. I married one of them, but I’m here as a UI designer, like I’ve poured over these videos.

(26m 9s) Tim Chaten:

Now.

(26m 9s)

Right.

(26m 24s)

Yes. Yeah. Yeah. So, I guess I should start by asking, is there anything at all you can share about the physical experience of using this thing? Comfortability, expectations where they met, like, I don’t, anything at all in the physical realm, I’m, that’s probably,

(26m 24s) Clarko:

And so not here as the spy, I’m just here to like share enthusiasm because I’m excited about working on it.

What can you share about the physical device?

(26m 54s) Tim Chaten:

that’s probably the one realm that’s probably the hardest to walk around.

(26m 58s) Clarko:

Yeah, I think, um, because like they told the press, even in that those first few demos that like, that they were still working on the light seal or something, or they didn’t have all the light seals available, there was.

(27m 8s) Tim Chaten:

Yeah.

(27m 9s)

Yeah, they’re still making money.

(27m 12s) Clarko:

So there’s a ton of that stuff where I’m just like, well, I got what I got.

(27m 16s)

And it was, you know, I don’t know if it was my size.

(27m 20s)

I have absolutely no insight into whether it was correct.

(27m 24s)

And I don’t, I don’t have corrective lenses.

(27m 26s)

So I didn’t, you know, there’s.

(27m 28s)

I don’t know how representative the experience is to what we’ll ship.

(27m 32s)

Cause in, in the back of my mind, I’m like, okay, the hardware seems done, but if they say all these accessories aren’t finished, then is that what’s, is that what the shipping, um, deadline is all about?

(27m 45s)

Are they like, because hardware has pretty long lead times.

(27m 50s)

You can’t just turn that all around super quick.

(27m 52s) Tim Chaten:

Yeah, is the OS the thing that needs polishing, or the hardware, or a little bit of both?

(27m 57s)

Probably a little bit of both.

(28m) Clarko:

I’ll say it right off the bat, like my expectations were really high because of what they showed in the keynote and my expectations have stayed high.

(28m 8s) Tim Chaten:

And what the press has been saying as well, yeah, all the press, not a single press was like, this wasn’t good, like everyone was like, whoa.

(28m 15s) Clarko:

Yeah. Yeah. It’s technologically like just on paper is so impressive. And then everything we’ve seen for the functionality and the visuals also super impressive. And so I am impressed.

(28m 30s)

There are parts of it that just really fascinate me. And this is, you know, I can’t not be a full time designer. It’s just, you know, what I’ve been doing for my whole life. But the relationship between your eyes and.

Eye Control Ramifications

(28m 45s) Clarko:

Your gaze and the user interface is really fascinating.

(28m 46s) Tim Chaten:

Yeah Right Yeah, it’s interesting conceptually you can say that and until you get to use it it’s like oh

(28m 48s) Clarko:

Everybody knows that your eyes are the pointing device in this thing,

(28m 52s)

but that’s actually a really big deal because.

(29m 3s)

Well, it just, it upends so much conventional wisdom about user interface design, because every computer I’ve ever used, the cursor is controlled by your hand.

(29m 13s)

And I was talking about the Quest 2 where it like feels like a laser pointer, because you’re holding this cursor and then you’re pointing it at something that’s like 30 feet away.

(29m 22s)

And you know how when you hold a laser pointer, it’s very wiggly when you point it against the opposite wall.

(29m 26s)

Like that feels weird in Quest.

(29m 29s)

And you know, otherwise you’re using keyboards and mice.

(29m 33s)

and styluses or just your bare finger.

(29m 35s)

So you like, you look at the screen and you see something you want to click and then you move the pointer manually with your hand to click the thing.

(29m 42s)

And with Vision OS, cause your eyes are the pointer and the screen is everywhere.

(29m 48s)

Now you just look and that’s it.

(29m 51s)

Like you, there are no gross movements.

(29m 54s)

You’re just, you can just look at something and click.

(29m 56s)

And that is, that is revolutionary.

(30m)

And it’s like, I know that it’s also a cool gimmick.

(30m 3s)

But it’s, it changes everything about how you design user interfaces, because something that they’ve really nailed into us for some reason at school was, um,

(30m 12s)

Fitts’ law, which is an equation that quantifies essentially how hard it is to point at stuff.

(30m 19s)

It’s you take like relative measurements of size and distance and time.

(30m 24s)

And it’s just like, oh, okay.

(30m 25s)

Things that are big and close are easier to hit than things that are far and small.

(30m 29s)

And that applies whether you’re throwing a baseball at a barn or whether.

(30m 33s)

You’re shooting at a target or whether you’re using a mouse.

(30m 36s)

And that, that just seems like it’s completely abstracted away in this model of eye tracking, because if you can see it, you have hit it.

(30m 44s)

It’s the effort.

(30m 46s)

The effort just seems built in to our eyes.

(30m 49s)

We don’t think about looking at something as a challenge.

(30m 53s) Tim Chaten:

• Right, discoverability.

(30m 56s)

I also said the challenge for a while of,

(30m 59s)

is this tappable?

(31m)

Is this a long press do anything?

(31m 3s)

Does this get even harder as a designer to design an app to know, to tell a user,

(31m 9s)

this thing you’re looking at is tappable or taps the word we use when we tap our fingers together.

(31m 15s)

Is that the right nomenclature?

(31m 17s) Clarko:

• Yeah, that’s the official terminology.

(31m 18s) Tim Chaten:

Yeah.

(31m 19s)

Okay, so let, yeah.

(31m 19s) Clarko:

I keep thinking of click when I turn in terms of mice and stuff, but yes,

(31m 23s) Tim Chaten:

Yeah, but like designing an app around,

(31m 23s) Clarko:

tapping is the official terminology.

(31m 26s) Tim Chaten:

like does the OS convey what is tappable as you’re looking kind of like Apple TV does with its little shimmer effect on everything you look at or like, how’s that?

(31m 37s)

Like, how’s that?

(31m 38s) Clarko:

• Yeah, I mean, they talk about that at length in the keynote, and it’s a conceptual leap.

(31m 45s)

There is this argument on software teams about buttons and links looking clickable enough,

(31m 50s)

’cause otherwise, how will the customer know that they should click it?

(31m 52s) Tim Chaten:

[laughs] Right. Yes.

(31m 54s) Clarko:

Because sometimes there’s text and icon on screen for no reason at all, and you shouldn’t click those,

(31m 58s)

but you should click the Buy button,

(32m)

and ’cause Vision OS has that highlight effect where it puts a bit of shimmer on it and a shape around it.

(32m 6s)

If it’s just text, it’ll put a.

(32m 8s)

Shape around it or an icon.

(32m 11s)

So when your eyes land on something, it gets that highlight effect.

(32m 14s)

And like I was saying, your eyes are the pointer.

(32m 16s)

You’re never not looking at something.

(32m 18s)

So every time you look at something, you know, instantly, can you tap on it or not?

(32m 24s)

There’s absolutely no question.

(32m 26s)

And that really feels like it changes the job of the designer because you’re not arguing about like whether this should be more red or like bolder or like whether it should have a different shaped border around it.

(32m 38s)

Because you’re trying to attract attention.

(32m 40s)

It’s just, or the, Oh, wow.

(32m 43s)

The age old like web design argument about should a link be underlined or not.

(32m 46s)

And it’s like the affordance, the affordance of tapability is just built in here.

(32m 53s)

You can’t not have it.

(32m 54s)

And it makes button design, not that button design is a specialty, but it makes the idea of button design just kind of moot.

(33m 2s)

And I get, I get the impression that a lot of iPad ports or, you know, iPad apps

(33m 8s)

ability mode will be kind of over the top looking because all the, uh, vision OS apps that we’ve seen, they look very kind of austere.

(33m 16s)

They’re very sparing with like with color and, and, and button shapes and stuff.

(33m 22s)

And all these iPad apps with huge blue rectangles in them will just like, look kind of like a clown show unless everybody goes the far opposite direction and all apps start looking like, like Vegas slot machines.

(33m 24s) Tim Chaten:

laughs Yeah.

(33m 33s)

Right. Yeah. Does the size of the button, do you think that’ll like larger buttons kind of help the user like point their eyes that direction to this is one of the primary buttons to be looking at while I’m using this app? I’d imagine size would be a factor.

(33m 33s) Clarko:

Like it could go either way.

(33m 52s)

I mean, there’s definitely a matter of like trying to catch your attention in the periphery.

(33m 57s)

That is always the case.

(33m 58s)

And it’s funny, there’s also like the concern for people designing Vision OS apps is screenshots.

(34m 5s)

Because screenshots do not respond to your gaze, nor do demo videos.

(34m 8s)

So it’s like, your app could look very boring in a screenshot, but that could be totally fine in real life, because the buttons always light up when you look at them.

(34m 16s)

So there’s a real balancing act there.

(34m 20s)

There is a whole thing about it.

(34m 22s)

button sizing in Vision OS that I can talk about.

(34m 26s) Tim Chaten:

Yeah, yeah, please do.

(34m 27s) Clarko:

They talk about it in the design guidelines that you need buttons to be a hit target of 60 points.

(34m 35s) Tim Chaten:

Okay. Yeah, 60 points sounds tiny to me because I’m used to like, you know, you know, podcast artwork is whatever, a thousand, but you know, it’s like.

(34m 38s) Clarko:

Points is a totally arbitrary metric.

(34m 47s)

Yeah. Well, in the first days of iOS, a point was a pixel. There was just a one. We used to call, we would just call them pixels. We didn’t call them points. Then when things went to retina, we had to add this level of abstraction. So it’s like, it’s no longer a pixel. It’s a point. It represents a two by two pixel square in old fashioned terms.

(35m 8s) Tim Chaten:

Right. Yeah.

(35m 9s) Clarko:

Now we’re talking about a UI that is not even measured in pixels. It’s measured in degrees of that.

(35m 17s)

field of view.

(35m 19s)

And so I don’t know what the conversion is specifically of degrees to points, but when they say it needs to be 60 points, they’re doing that because they need that for hit targeting.

(35m 31s) Tim Chaten:

Right, and the the windows will dynamically change depending if you put the default Focal distance versus trying to bring it closer or further away

(35m 41s) Clarko:

Exactly. Yeah, because on an iPhone, the recommended hit target size is 44 points.

(35m 46s)

That’s like the smallest button they think you can get away with. But that is also,

(35m 51s)

that’s not an aesthetic decision, that is a reality of firmware engineering. Because you’re working with touch panels, they have this capacitive grid, and they do a bunch of heuristics in code to just take this sloppy analog input from a touch panel and turn it into an event for the UI software. And that’s, you know, the joys of firmware engineering. That’s

(36m 11s)

dealing with garbage input, and you’re trying to make useful output in it. But in Vision OS,

(36m 15s)

they’re doing that with an infrared camera and ML models that track your eye movements.

(36m 21s)

And so it’s just a fundamentally different technology with totally different constraints.

(36m 25s)

And so they just obviously did a ton of testing, and they’re just like, all right,

(36m 29s)

within our margin of error, 60 points is what we can say is reliable for a button to look at.

(36m 34s)

And it really makes me wonder about future generations of IR cameras. I don’t know if that’s a field that gets a lot of.

(36m 41s) Tim Chaten:

[Chuckle]

(36m 41s) Clarko:

Uh, advancement in it year over year, but like future generations of camera,

(36m 42s) Tim Chaten:

Yeah.

(36m 43s)

Mm-hmm.

(36m 45s) Clarko:

future ML models, obviously that is a field that gets a ton of investment in it year over year.

(36m 49s)

Could you radically improve eye tracking?

(36m 52s)

And when you do, what do you do with that improvement?

(36m 55s)

Like, do you just make all the windows a little smaller?

(36m 57s)

Do you like change the distance multiplier on that dynamic scale you’re talking about where you drag your window closer and further that it gets bigger and smaller, like, or do you do something entirely new that I can’t even imagine?

(37m 5s) Tim Chaten:

• Yeah.

(37m 5s)

Right.

(37m 10s)

• Yeah, it’s curious.

(37m 11s)

The eye tracking stuff, it’s in PSVR too,

(37m 15s)

and it’s great for games.

(37m 17s)

So I really hope one day Apple adds an API

(37m 22s) Clarko:

[Chuckles]

Text Selection?

(37m 26s) Tim Chaten:

One thing I’m curious about, I’m not sure if this has been publicly shared, is so right now on iPad OS, when I have my cursor up and I’m hovering over text, my cursor transforms into a thing that lets me select the text and edit it, you know, highlight it.

(37m 40s)

In Vision OS, as you’re looking at text, that is text you’ve typed into the interface, is there any notion of this way of highlighting this text, or is that reserved for perhaps if you bring in an external mouse?

(37m 56s)

mouse or trackpad that at that point.

(38m) Clarko:

That is a good question and something I can’t answer out of ignorance, not because of,

(38m 5s)

but they did talk in one of the videos,

(38m 10s)

maybe it was State of the Union,

(38m 12s)

about this like taking your cursor and doing like a bit of like a pinch gesture on it to make it a precision cursor for drawing.

(38m 18s) Tim Chaten:

Okay. Yeah.

(38m 20s) Clarko:

And they were demoing that in free form.

(38m 20s) Tim Chaten:

Okay. Yeah.

(38m 23s) Clarko:

They’re kind of whiteboard everything app.

(38m 27s)

And so there was an element to that,

(38m 29s)

but it’s also.

(38m 30s)

Interesting that in iPadOS, I think that text selection thing in general is really limited to editable text, with the exception of web pages.

(38m 38s) Tim Chaten:

Right.

(38m 39s)

Yeah, Safari gets it all day long now.

(38m 41s) Clarko:

And I don’t understand the distinction there.

(38m 42s) Tim Chaten:

And screenshots, I think, gets it as well.

(38m 44s) Clarko:

Because like I work in SwiftUI.

(38m 49s) Tim Chaten:

I’m pretty sure if I open photos.

(38m 56s) Clarko:

Yeah, because yeah, I work almost exclusively in SwiftUI and SwiftUI has this modifier that you can put on text to say that it’s selectable.

(39m 4s)

And when you do that on Mac, you get exactly what you’re expecting, where you can just select like character by character.

(39m 6s) Tim Chaten:

Mm-hmm. Yeah. Yeah.

(39m 9s) Clarko:

You do that on iOS, it’s just a long press with the copy button.

(39m 13s)

Like you don’t get you don’t get that cursor text selection behavior when you’re doing a SwiftUI and all of I mean, to my knowledge, the primary user interface development.

(39m 26s)

I’m not 100% sure that you’re allowed to just write UI kit or app kit software directly to Vision OS.

(39m 38s) Tim Chaten:

Yeah. Another thing I’m curious if they’ve shared yet is, so audio on iPadOS can be frustrating at times where it’s just a single app that has full audio control and if you start something else it takes over. Have they shared in this spatial computing environment if multiple apps could have audio and maybe even if, maybe even those apps would be positionally sending the audio to you so you’d be able to hear where the app’s coming from?

Multiple Audio Sources?

(40m 7s) Clarko:

They, there are several dub dub sessions about spatial audio and I have not watched them because it’s not something that is part of my like day to day work.

(40m 11s) Tim Chaten:

Yeah.

(40m 16s) Clarko:

Um, so there’s definitely spatial positioning of audio.

(40m 20s)

And I know, uh, Steve’s Troughton Smith has talked a bit about this in relation to his radio app broadcast and, and I couldn’t answer it for you, but I think like somewhere between the dub dub sessions and.

(40m 27s) Tim Chaten:

Broadcasts. It says that. Yeah. Yeah.

(40m 32s)

Okay.

(40m 35s)

Yes. Yeah. Yeah. I am. Yeah. I’m super curious. Like, it feels, a lot of people have said the iPad OS is like, you know, the father of Vision OS. Do the OS’s feel related? Or, like, it seems like Vision OS is like the father of Vision OS.

(40m 37s) Clarko:

Steve’s toots, there’s an answer there.

iPadOS vs visionOS

(40m 57s) Tim Chaten:

Vision OS has many more windows you’re working with, this whole new interface.

(41m 4s) Clarko:

You can see just from the visual design of the apps, like the Messages app and the Freeform app,

(41m 10s)

and all the apps that they have in Vision OS look very, very much like their iPad counterparts.

(41m 18s) Tim Chaten:

Yeah.

(41m 18s) Clarko:

And, you know, that’s not by accident. And the multitasking that also fascinates me about Vision OS feels like iPad Stage Manager, but with so much more screen real…

(41m 19s) Tim Chaten:

Right.

(41m 20s)

Yeah.

(41m 20s)

Right.

(41m 34s)

Yeah, you have unlimited windows by already.

(41m 34s) Clarko:

state that you’re not like constantly being kicked by Windows. Yeah. Because, you know,

(41m 41s)

you know, macOS multitasking, you have one foreground window. And, you know, you’ve got your dock and you’ve got your menu bar and got your app switcher. All these things are built around this idea that there has to be one foreground app because it controls the menu bar.

(41m 56s)

And there has to be one foreground window because that accepts keyboard input. Yeah.

(41m 59s) Tim Chaten:

And it controls the keyboard shortcuts as well, yeah.

(42m 4s) Clarko:

iPadOS has that to an extent. It has keyboard focus if you have a hardware keyboard. Because,

(42m 10s) Tim Chaten:

Yeah, when it works. Sometimes the keyboard focus is like, “Why is this keyboard working in the other app?”

(42m 11s) Clarko:

yeah. Yeah. Not my favorite aspect of iPadOS, but I understand that I’m also in a minority of people who use it with a hardware keyboard all day every day. But if you’re just… I have a

(42m 16s) Tim Chaten:

Yeah, it’s getting there, but yeah.

(42m 31s) Clarko:

a workspace on my iPad, which is my social media.

(42m 34s)

And I have Instagram and Mastodon and Threads all side-by-side, because two of those are just iPads.

(42m 34s) Tim Chaten:

Mm hmm. Yeah. Right. Yeah. Yeah.

(42m 41s) Clarko:

Two of those are just iPhone apps.

(42m 43s)

They have to be very narrow windows.

(42m 45s)

So I just throw a third one in there and make it a narrow window and have them all perfectly side-by-side.

(42m 49s) Tim Chaten:

Mm hmm.

(42m 49s) Clarko:

And you can just interact with them all.

(42m 51s)

You’re not bringing one forward.

(42m 53s)

You’re just touching what you need to touch.

(42m 55s)

And the multitasking is just invisible.

(43m) Tim Chaten:

Yeah, because in Vision OS, so in Mac, it’s been a while since I’ve actually used it.

(43m 5s)

Things go gray for the inactive app, right?

(43m 8s) Clarko:

Mm-hmm. Yeah

(43m 8s) Tim Chaten:

So in Vision OS, as you’re looking at things, that becomes the primary app just by your look.

(43m 15s)

I’d imagine if you look at an app, that’s where the keyboard focus would go if you use an external keyboard.

(43m 21s)

And as you change where you’re looking, that’s, yeah, if you start a click and drag with like an external keyboard or mouse, and then you look at a different window, you’ll be able to pull it into the app.

(43m 31s)

like it feels like it should.

(43m 36s) Clarko:

Yeah, and it feels like it should just be iPadOS, like, you know, to your point,

(43m 41s) Tim Chaten:

Yeah You’re right it less powerful

(43m 42s) Clarko:

the iPadOS seems like the, you know, the, the, it’s weird to call it the dad because it’s, you know, this smaller, like, but you know, the, the progenitor.

(43m 50s)

Um, because I mean, Alexander McGinnis, like talked about this at length in the keynote and they had all of that, all that B-roll of people doing multitasking.

(44m 1s)

And there’s that, that guy at the desk with like three or four apps all around

(44m 6s)

and none of them is dim, none of them is like, none of them is noticeably background and, you know, with our experience with iPadOS, it’s like,

(44m 7s) Tim Chaten:

Yeah, that’s a good point. Yeah, right.

(44m 14s) Clarko:

all right, I just, I can just scroll this window.

(44m 16s)

I can just scroll this window.

(44m 17s)

I can tap this button.

(44m 18s)

And it’s like, nothing ever jumps to the foreground.

(44m 20s)

And in one part of the keynote, the guy drags a, uh, uh, 3d model out of a messages thread and drops it on the table in front of him.

(44m 29s)

And that’s a new window.

(44m 30s)

That’s just, that’s a volumetric window instead of just being a 2d window.

(44m 33s)

And nothing else happened.

(44m 36s)

It just reacted to like suddenly not being the focus of, of his input.

(44m 40s)

So it really does seem like it’s based on iOS, iPadOS for that, for that reason.

(44m 46s) Tim Chaten:

Yeah, right.

iWork and 3D Printing/Sculpting

(44m 47s) Tim Chaten:

Apple recently updated iWork with USD support.

(44m 51s)

Is this just Vision OS prepping in your mind?

(44m 56s) Clarko:

I’m not sure. I mean, we’ve had USDC support in Safari for a really long time, and they’ve had this AR Quick Look built into iOS for a really long time. So it just seems like something that the OS is capable of, it’s reliably capable of it, going back several versions. So if you need to add support to it for your app and your app supports older OSs, then go for it. But I’ve certainly been intrigued.

(45m 26s)

By the idea of just like including USDC as a media type in normal apps, like could Instagram let you upload USDC files and you just like scrolling through a feed of 3D objects? That’d be great. But I don’t like I don’t know what effort is required to do that. And if you’re a multi platform app, obviously there are limits on what you can do. Like if you are Instagram, you have to build the USDC viewer for the web and for Android and for every platform that you ship on.

(45m 29s) Tim Chaten:

Right.

(45m 37s)

Right. Yeah.

(45m 53s)

Yeah, USDZ will be, with Vision OS, will be, I think just the apps that create those on Vision OS should be just a lot of fun to just to play around with and build stuff that I wonder if the 3D printing machine market will just kind of get bigger along with

(46m 18s) Clarko:

Yeah, there are certainly like VR apps, like is it called Light Sculpt, like there are sculpting apps in VR that feel like they would do even better if you had, you know, if you’re using hand tracking instead of controllers.

(46m 33s) Tim Chaten:

be so cool you start with like a block of ice or whatever it is and you have hand tracking and

(46m 42s) Clarko:

• Yeah, and then I’m not, I’m speaking out of turn for USDZ,

(46m 47s)

but I get the impression that you can either create objects that you look at from the outside,

(46m 52s)

or you can create like shelves that you stand inside and look out at.

(46m 55s) Tim Chaten:

Yeah.

(46m 55s) Clarko:

I don’t know if that’s how they make their environments on Vision OS, but like, it would be great to just be sent a little file and you open it,

(47m 4s)

and then you’re standing in a place,

(47m 6s)

you’re standing in a new environment.

(47m 8s)

Like that would honestly like being.

(47m 12s)

I would to visit anywhere or places that never existed or places that no longer exist or just, you know, meditate by a stream that would be a three hour drive from where you are would be wonderful.

Capturing Scenes for Vision Pro

(47m 24s) Tim Chaten:

It would be super cool. So iPhone 15 pro capture spatial video. It’d be cool if there was a mode to like, I hiked to the top of one of my favorite mountains to hike and I capture whatever kind of,

(47m 39s)

maybe an app guides me through a capture process, an Apple app to capture a scene.

(47m 43s)

And I can just capture this, this whatever scene, um, like almost like panoramic,

(47m 49s)

but like a video version. Yeah, and I can then use that.

(47m 50s) Clarko:

Yeah, like a spherical pan.

(47m 54s) Tim Chaten:

That’s cool.

(48m) Clarko:

Yeah, I don’t know if you’ve ever used WebXR formats on any of your other headsets.

(48m 7s)

It’s stereoscopic video, but it’s very jarring because you don’t control where you look.

(48m 13s) Tim Chaten:

Oh, yeah.

(48m 14s) Clarko:

So everything is 3D, but the director is very much in charge.

(48m 18s)

So I tried doing some roller coasters and some rides at Disneyland that I’ve never been on because I haven’t been back there in years.

(48m 25s) Tim Chaten:

Yeah It’s not following your head, yeah The gestures Apple watch it took them many years to figure out What the side button should do and all this kind of stuff from what you’ve seen with the gestures that they’ve you know Shown off to the public does it seem like a natural way to interact with everything or are they missing?

(48m 26s) Clarko:

And it’s, it’s really difficult because you’re not in charge.

(48m 30s)

of anything, yeah.

The Gestures

(48m 52s) Tim Chaten:

Key things that you know future they should add some future

(48m 55s)

gestures to the, to what the slide.

(49m) Clarko:

They seem to me like they’ve been very conservative, which is fair for like, you know, an Apple 1.0 product.

(49m 6s)

You’re, you are conservative because you can’t wait to see what we do with it, you know, is the phrase.

(49m 12s) Tim Chaten:

Yeah, iPhone OS, or iPhone firmware 1 or whatever, did not have copy and paste.

(49m 19s) Clarko:

[laughs]

(49m 19s) Tim Chaten:

It took a while to figure out a lot of stuff.

(49m 21s)

It had the magnifier glass, I think, to change your text position.

(49m 27s) Clarko:

Yes, and so what are their published gestures?

(49m 32s)

They have pinch, obviously, and they have,

(49m 35s)

oh, sorry, pinch, tap.

(49m 36s) Tim Chaten:

Yeah.

(49m 36s) Clarko:

They have tap, and then they have pinch and unpinch,

(49m 37s) Tim Chaten:

Tap and there’s pinch.

(49m 40s) Clarko:

the zooming in and out gestures that they show when you’re using photos, and a few others,

(49m 47s)

but it seems like a really limited set,

(49m 49s)

and I have to assume more will come,

(49m 52s)

but I also assume that since they’ve given developers such.

(49m 57s)

Deep access to the hand tracking APIs that they’re waiting to see if developers dive in there and invent stuff.

(50m 4s)

Because they’ve done that with the advanced gestures on iPadOS.

(50m 9s)

They didn’t give us any really interesting gestures like undo and redo on iPadOS for years.

(50m 19s)

So many years.

(50m 19s) Tim Chaten:

Yeah, and yeah, it still works on iPad Pro. 13-inch iPad Pro, you can shake to undo,

(50m 21s) Clarko:

Besides shake, which you never want to do with an iPad.

(50m 24s)

And yeah

(50m 27s)

Yeah, so I think I feel like they waited I feel like they waited for a certain class of app, especially drawing apps where you’re, you know, you have your hands are busy and you are not shaking the thing and you are doing a lot of undoing and undoing. I think they let the market invent a lot there. And then then they’re just like, okay, let’s standardize this now and let people just ride the standard.

(50m 27s) Tim Chaten:

and I occasionally look very silly doing that. Um, scroll, er, um…

(50m 51s)

Yeah, and they took scroll to, um, what is it? Scroll to refresh.

(50m 57s) Clarko:

Mm-hm, pull to refresh, yeah.

(51m 2s)

And I get the feeling they’re gonna do a lot of that here,

(51m 6s)

assuming that there are a good body of developers inventing interesting gestures.

(51m 12s)

Because, yeah, the list of gestures they have.

(51m 15s) Tim Chaten:

There is a capacity for developers to include their own custom gestures that are not system-wide. Is that a feature of the OS?

(51m 21s) Clarko:

Yeah, the hand tracking, the hand tracking APIs are extensive.

(51m 21s) Tim Chaten:

That’s exciting to hear. Yeah, that’s awesome. Right.

(51m 26s) Clarko:

Yeah, I don’t know how hard it is to work with.

(51m 31s)

But certainly if you’re if you’re someone like the, you know, VR sculpture apps and your whole user interface is using your hands,

(51m 43s)

you’re going to dive deep into that to see what you can do.

(51m 45s) Tim Chaten:

Yeah, create a little blowtorch with your, you know, grab the.

(51m 46s) Clarko:

[laughs]

(51m 48s)

[laughing]

(51m 52s)

• But yeah, like of the published gestures they have,

(51m 54s)

you know, there’s no undo gesture,

(51m 56s)

which I obviously desperately want on every single platform that I use because I always need to undo the stupid thing I just did.

(52m 3s)

And those I assume will come,

(52m 6s)

but they’re just being conservative out of the gate.

(52m 9s) Tim Chaten:

And I wonder how deep the hand gesture API is, like could a developer, I don’t know, you sell this little like hardware thing that’s like a fake sword or something.

(52m 24s) Clarko:

I think we need that for Beat Saber and for Vader Immortal Lightsaber Dojo.

(52m 27s) Tim Chaten:

Right?

(52m 29s)

Yeah, just something to hold it doesn’t even have to be Yeah, it doesn’t even have to be Bluetooth. Just something just to hold like yeah I wonder if it can work with that kind of stuff Yeah, any other just general thoughts about Vision OS Before we move on to your app in particular Yeah

(52m 29s) Clarko:

I don’t think I can play those games with empty hands.

(52m 32s)

Yeah.

(52m 38s)

Little hilt.

Any other general thoughts?

(52m 47s) Clarko:

Yeah, I mean, we started talking about expectations in the beginning, and it’s funny because Apple is all about managing expectations.

(52m 57s)

They have that reputation for shipping something small and focused and polished and then iterating, but.

(53m 3s)

They sell that thing.

(53m 7s)

Like they show you what they made and then they sell it to you.

(53m 10s)

And so, you know what they showed at dub dub isn’t just a fantasy.

(53m 13s)

It’s not concept art.

(53m 15s)

It’s a real product, but.

(53m 17s)

They only showed it seemingly in office and home entertainment workloads.

(53m 25s)

There was, there wasn’t a lot outside of that.

(53m 30s)

And I’m, I’m going to call sitting in a plane seat, a home entertainment situation.

(53m 34s) Tim Chaten:

Right. Yeah. I desperately hope this device operates on like a porch outside in the backyard,

(53m 42s)

like, but we’ve not seen that.

(53m 43s) Clarko:

Me too, me too, but they’re really right.

(53m 46s)

So they’re setting, they’re setting that expectation and you have to assume because of how good they are at this job, that they’re doing it on purpose.

(53m 55s)

Like it’s a Mac for your face.

(53m 56s)

And it’s silly after the keynote, the very first thing I imagined it for was rally driving.

(54m 4s)

And I’m not a race car driver.

(54m 6s)

I don’t like, I don’t know why that jumped to mind, but like rallying for or anyone who doesn’t know is a motor sport where you’re.

(54m 13s)

Racing on urban or off-road courses and it’s a driver and a navigator because the courses are very complicated You need a navigator who’s sitting there looking at the map and then making hand gestures like, you know, turn right in 200 feet Something to that effect which is the opposite of office work But I thought coming right out of the keynote like hey take the navigator out of the car give the driver a cool AR heads-up display and Save 200 pounds of weight and like go faster and win a race. Maybe that’s against the rules I have no idea.

(54m 19s) Tim Chaten:

Yeah, uh, yeah.

(54m 40s)

Right.

(54m 40s)

Yeah.

(54m 42s)

No, it’s disabled.

(54m 43s) Clarko:

Like I said, I’m not a race driver, um, but Apple never showed anyone in a car.

(54m 48s)

It’s probably not safe.

(54m 50s)

So, yeah, so I just jumped to that assumption that it’d be possible.

(54m 56s)

And then Mike Rockwell said at the talk show in San Jose, how hard it was to make it work on a plane because of the, uh, the IMU, the inertial measurement unit that like tracks the gyro and accelerometers and stuff to determine how much you are moving in space.

(55m 11s)

And if you’re on a plane, you’re always moving in.

(55m 13s)

So they’re doing some kind of inertial compensation there.

(55m 16s)

That’s obviously very complicated, but yeah.

(55m 18s) Tim Chaten:

And there’s like a plane mode I think you…

(55m 23s) Clarko:

So I have no idea if plane mode works on a, on a bus, but you know, there’s.

(55m 28s)

There’s a lot of Apple employees who are riding a bus to work every day and they probably want this product.

(55m 34s)

Um, and, and like you said, like, can you do it in your back deck?

(55m 37s)

Is direct sunlight even?

(55m 38s)

Okay.

(55m 39s)

because like I played lightsaber dojo in the backyard.

(55m 43s)

with quest two and the tracking was all off. It was,

(55m 46s)

I had to stop very quickly. So expectations are high.

(55m 50s)

Like it’s obviously a very, very polished, like very mature looking product,

(55m 55s)

but expectations are also very narrow. We’re talking like,

(55m 58s)

it really seems like you want to be standing or sitting indoors and we don’t know where the edges are of this product yet,

(56m 5s)

which is very frustrating as people who are trying to invent new things for it because they’re obviously really proud of it as a work tool.

(56m 7s) Tim Chaten:

Yes.

(56m 8s)

Yeah.

(56m 13s) Clarko:

It’s an entertainment platform, but we all see potential for it in mobile computing, or like mobile spatial computing.

(56m 20s)

But we don’t even know if it is mobile.

(56m 22s)

We know that it has two hours of battery life, and that’s all we know.

(56m 25s)

We don’t even know if it has cellular or GPS.

(56m 27s) Tim Chaten:

Yeah, I’d imagine Cellular or not, they probably would have shared that tidbit.

(56m 32s)

Or maybe, who knows, maybe that comes in the January reveal and they reveal all the specs and they’ve just shared a little bit for now. Who knows?

(56m 35s) Clarko:

Who knows?

(56m 36s)

Yeah? I guess we don’t even have cellular max yet, so that seems like maybe a leap.

(56m 40s) Tim Chaten:

We do not.

(56m 43s)

It might be sad to say, but I kind of expect this to get Cellular before the Bax do.

(56m 51s) Clarko:

So, yeah, like I’m, I’m really excited and really impressed by this whole product, but I’m also trying to ground myself in reality, which is like when, when we talk about my app, it’s going to be silly because that is the most grounded in reality.

(57m 5s)

It is not like a groundbreaking outlandish app.

(57m 9s)

It’s a normal office productivity app, but I’m trying to rein in my enthusiasm to stay within the bounds of what they actually showed, because what they showed is cool and I’m excited about it, but the mind just races.

(57m 21s)

at possibilities that we don’t even know if they’re possible.

(57m 25s) Tim Chaten:

Yeah, definitely Did you end up getting a 15 pro to start gathering spatial videos of your your two-year-old pup?

(57m 35s) Clarko:

Not yet. Part of me is concerned about coordinating the replacement of cables in our lives.

(57m 42s) Tim Chaten:

Ah, yes.

(57m 42s) Clarko:

The car in particular, we use wired CarPlay, and that’s lightning.

(57m 43s) Tim Chaten:

Yep.

(57m 46s)

Yeah.

(57m 47s) Clarko:

So it feels like we’ll need to coordinate that change between me and my wife.

(57m 47s) Tim Chaten:

Yes.

(57m 50s)

Yeah.

(57m 52s) Clarko:

I’m on the replacement plan thing, and part of me is also juggling this load of old devices that I have to keep around for software testing.

(58m 1s)

I’m like mentally wondering what version of the OS each.

(58m 5s)

gets upgraded to as its last hurrah.

(58m 8s)

And so my newest testing device is a 13 mini, which I love and miss.

(58m 8s) Tim Chaten:

Yes.

(58m 9s)

Yeah.

(58m 14s)

Oh yes. I hope they make a six.

(58m 15s) Clarko:

Uh, and so that’ll probably, yeah, so like that’ll probably take us to iOS 20.

(58m 25s)

That’s not a number I expect to say out loud, but like, so the 13 mini will probably get iOS 20 and now I have a 14 pro.

(58m 26s) Tim Chaten:

Yeah. Right.

(58m 35s) Clarko:

So should I just skip a year and like then manage these, all these operating system releases?

(58m 40s)

It’s such weird calculus to do as a developer, like what hardware do you keep around?

(58m 45s)

And when does it have to be retired?

(58m 47s)

Cause it’s not getting updates.

(58m 49s) Tim Chaten:

Yeah, it’s uh, it’s interesting that being a software developer like long it must be so much more painful that phones have gone up and up in price versus like $600 original iPhone that’s extravagant you know, yeah, but True yeah

(59m 6s) Clarko:

Yeah, although then again, like going from having to build my own PCs as a kid to like being able to just buy a super great laptop and it’s like comparatively speaking, the

(59m 18s) Tim Chaten:

Yeah. Is spatial video capture something you’re excited for with the headset when you get it?

(59m 22s)

Is that something? Yeah.

(59m 24s)

Yes. I know, right?

(59m 25s) Clarko:

Yes.

(59m 26s)

I mean, like for the dog reasons, I had such a, like I, I, yeah, I am so intrigued by it and, uh, I think that the iPhone 15 and future generations of, of iPhone are certainly the right device to be capturing that on every day.

(59m 30s) Tim Chaten:

Yeah. I’m excited to, you know, capture my CAD.

(59m 48s)

Yes, it will be nice, I think, for some captures to be on the headset so your hands are free and you can capture things with, in situations that maybe an iPhone would actually be harder to do.

(59m 59s)

But yeah, most of the time.

(1h 4s) Clarko:

Yeah, I’m also very curious about the role of the shooter’s perspective in this video capture,

(1h 9s)

because last night I shot a video of my dog sleeping for 30 seconds because his little paws were wiggling, and I shot that from essentially couch level, like my own thighs, and like that’s a weird perspective if I’m looking at it in 3D because that’s that would mean that I’m like squatting. And is a video shot by a very tall person in a headset very different than a video shot by a very short person.

(1h 15s) Tim Chaten:

Yeah, right.

(1h 33s)

Right, yeah No, yeah, and that’ll be yeah, that’ll be interesting to play around with once the headset comes out to try to expeed What’s the different experiences like how you’ll need to learn like?

(1h 34s) Clarko:

I don’t know what to make of that yet.

(1h 44s) Tim Chaten:

How is it what a best practice or shooting spatial video?

(1h 48s)

You probably don’t want to be doing anything too jarring just like slow moving

(1h 54s) Clarko:

Yeah, and I’ll be intrigued about future developments in that format as well, because we’ve come pretty far just on JPEGs and then all this like the Heave sidecar data with live photos and depth maps and all that stuff.

(1h 1m 8s)

So there’s, I don’t know what this format is that they’re shooting in, but they’ve got freedom to explore and add stuff before it gets like codified by the IEEE.

(1h 1m 17s) Tim Chaten:

Right Yeah, and I’m super curious next year when this rolls out if people will figure out ways to convert existing 3d footage into Apple’s formats. I’ve got I don’t no idea where the originals are. I’ve got some old GoPro footage. I shot in 3d back like a decade ago

(1h 1m 37s) Clarko:

Actually, you know what I’d be super, super interested in seeing is digitizing old Viewmasters and those old things that they had in the Nickelodeons, like the 3D things.

(1h 1m 54s)

They have some in Disneyland, these very old machines you put a dime in and look through.

(1h 1m 55s) Tim Chaten:

Yeah, right. Yeah. Yeah. Yeah, that’d be fun. Yeah. Yeah. I mean, I can imagine the people that have that stuff, if they can figure that out, that’d be a cool app to just sell, you know, this 3d content. Yeah. Yes.

(1h 1m 58s) Clarko:

And it was shot in 3D, but like a hundred years ago.

(1h 2m 1s)

It would be being able to travel back in time in 3D like that would be…

(1h 2m 19s)

Yeah, yeah. There’s definitely a market for just 3D content as a service.

(1h 2m 26s)

3D modelers, 3D artists will be very busy for the foreseeable future.

(1h 2m 26s) Tim Chaten:

Yes, for sure. Yeah. Yes.

Sponsor – Agenda 18

(1h 2m 34s) Tim Chaten:

This episode is sponsored by agenda. In this quick break, I want to share a bit about agendas big multi window update that happened earlier this year for both the Mac and the iPad, and is now very useful with stage manager on the iPad finally being in a happy place. These changes will also shine on the Apple Vision Pro with that it was being so perfect for with multiple windows. So before we get to all these awesome new multi window additions to the app, I want to make sure everyone knows about the new ability to navigate agenda very quickly with just the keyboard. By hitting option space, you’re now brought into a quick search box that by just searching for the name of the note or project, you can either hit enter to open the first result or arrow down to the result you want to open to enter to open that note or project. It’s super fast and is just so nice not needing to use the

(1h 3m 25s)

or touch the screen to jump to the note or project you want to work in next. So the huge thing added earlier this year was true multi window support for both the iPad and the Mac. Every note now has a little window icon. You’ll find this next to the gear icon at the bottom of every note on iPad. This button behaves a bit differently if you are in split screen mode versus stage manager in split screen. It’ll bring that window front and center and dim the background. The same behavior you’ll find in

(1h 3m 55s)

Apple’s Mail app. This is an awesome way to work if you do just want to focus on your single note and is the mode I work in when I’m writing up scripts in what I call iPad mode of allowing more focus than other environments provide. Just like in Apple Mail, you can manipulate this window with the triple dots on the top of the window allowing you to transform that window into a slide over window or use it in split-screen. At any time you can tap the icon on the bottom of the note to return.

(1h 4m 25s)

the note to the main agenda window where you started.

(1h 4m 28s)

If you are working in Stage Manager, hitting that window button will pop that note out as a brand new window that you can move and resize however you want.

(1h 4m 37s)

The sidebars of Agenda are hidden, but if you want to use that window for a different note, you can slide out the sidebars to move away from that note.

(1h 4m 45s)

Just like working in split screen mode, you can tap the window button on the bottom of that Stage Manager window to bring that note back into your original window you were working in.

Symbolsaurus

(1h 4m 55s) Tim Chaten:

This is a great system for using multiple windows in Agenda on the iPad and Mac, and something for sure worth trying out if you haven’t already.

(1h 5m 2s)

And I can’t wait to try this experience on Vision OS, as that OS screams multi-window and having a bunch of different windows scattered throughout your environment.

(1h 5m 14s)

So those are just a few new things worth checking out if you haven’t already in Agenda.

(1h 5m 18s)

If you haven’t tried Agenda, I’d really encourage you to give it a try.

(1h 5m 22s)

is free to download and use with no obtrusive ads.

(1h 5m 25s)

To unlock the full power of Agenda, give Agenda Premium a shot.

(1h 5m 29s)

One thing that Agenda continues to do that I love is their approach to the premium features and unlocks.

(1h 5m 34s)

If you ever decide to cancel your Agenda Premium subscription, you get to keep all of the premium features available when you are a subscriber, or you can now opt for their new Lifetime Unlock, which will unlock the full power of Agenda on all your Apple devices,

(1h 5m 48s)

including Apple Vision Pro, when that launches next year, with a single one-time purchase.

(1h 5m 53s)

To learn more, go to agenda.

(1h 5m 56s)

Download Agenda 18 today for free from the App Store.

(1h 5m 59s)

My thanks again to Jenna for sponsoring this episode of Vision Pros.

(1h 6m 3s)

Learn more at http://www.agenda.com.

(1h 6m 7s)

So, give me an elevator pitch for your app that is soon to be released at some point for Apple platforms,

(1h 6m 15s)

including Vision OS, when that comes out early next year.

(1h 6m 19s) Clarko:

Sure, my app is called Symbolsaurus, and it is a thesaurus of iconography.

(1h 6m 27s)

For anyone who’s never used the SF Symbols icon set, it’s a library that Apple publishes of thousands and thousands of icons.

(1h 6m 36s)

And designers and developers can use them instead of drawing new icons from scratch,

(1h 6m 40s)

which is labor intensive.

(1h 6m 42s)

And you know, if you’ve drawn one plus icon, you’ve drawn a million.

(1h 6m 45s)

Like, you don’t, no one should draw that icon ever again.

(1h 6m 47s) Tim Chaten:

Yeah. Yeah, these are things like, you know, these are things like the pause icon in like video players. Every imaginable…

(1h 6m 56s) Clarko:

Yes. Yeah, it’s seven or 8,000 of them right now. They just keep adding like 1,000 every year and they’ve been going for a while. And the icons are all named something like xmark.circle.fill.

(1h 7m 12s)

They’re named like with unique identifiers because you need to be able to call them from code and they don’t have any specific meaning. They just describe the shape. And that’s,

(1h 7m 22s)

I think that’s a philosophical stance on their part, because ExMa.

(1h 7m 26s)

Exmark.circle.fill, like is very often a close button.

(1h 7m 30s)

You’ve seen it used as a close button a hundred times, but it’s also a cancel button or a clear button.

(1h 7m 35s)

Like it’s the semantics are flexible.

(1h 7m 37s)

And so they’re not prescribing a name for what this symbol should be used as.

(1h 7m 44s)

And they’re very cagey about that.

(1h 7m 47s)

They kind of, they have some metadata in there.

(1h 7m 49s)

So if you search in, in Apple’s official app, if you search the word share, you You will get the arrow up.

(1h 7m 56s)

Square icon, which would otherwise, it would be impossible to find the share icon that we all use in every app.

(1h 7m 57s) Tim Chaten:

Mm-hm. Right, the share sheet, yeah.

(1h 8m 3s) Clarko:

Yeah.

(1h 8m 4s)

Cause it’s called like arrow dot up dot square or something.

(1h 8m 8s)

And so I manually collect all these real world labels, all this semantics and make it all searchable and all cross-reference.

(1h 8m 17s)

So like a dictionary, like a thesaurus, which is where the name comes from.

(1h 8m 20s)

Obviously I’ve compiled all this stuff into a reference tool.

(1h 8m 24s)

And so if you need a toolbar icon for.

(1h 8m 26s)

Edit button, you can just find one and it’s real world examples.

(1h 8m 30s)

And I’ll tell you where they all came from.

(1h 8m 32s) Tim Chaten:

Yeah, like in the close example, you have examples of it used for dismiss, cancel, close and clear and all the, you know, examples you may find this app in.

(1h 8m 32s) Clarko:

So you can judge for yourself and pick one with some confidence.

(1h 8m 49s) Tim Chaten:

And you also, you do tap into some third party apps as examples.

(1h 8m 54s)

I noticed Overcast as an example.

(1h 8m 58s)

It’s first third-party apps is this is the data.

(1h 9m 2s)

from just personal knowledge or is there some is there some way to easily find where this stuff is being used okay

(1h 9m 10s) Clarko:

• Not easily, definitely looked.

(1h 9m 15s)

No, this is a manual process.

(1h 9m 16s)

So it’s, I every week or so just sit down for a few hours and just pour through some apps,

(1h 9m 24s)

try and find every screen,

(1h 9m 25s)

try and find every context menu on iPad,

(1h 9m 28s)

try and find every single use of an icon.

(1h 9m 31s)

And with a combination of accessibility tools,

(1h 9m 36s)

like on the Mac, VoiceOver has a mode where you just hover on stuff

(1h 9m 40s)

to view the accessibility description.

(1h 9m 41s)

So you don’t have to navigate via VoiceOver like you would on iPad.

(1h 9m 47s)

So that the hover to reveal what the label for an icon is, is very helpful to me.

(1h 9m 52s)

And obviously all context menus on iPadOS,

(1h 9m 55s)

they have a label and an icon.

(1h 9m 57s)

So that’s very easy mapping.

(1h 9m 58s)

But I’m just doing it manually because this is literally what I’ve always done in my career.

(1h 10m 4s)

And it’s like, oh, I need to label this heart icon.

(1h 10m 7s)

Like, what do we call it?

(1h 10m 8s)

Do we call it favorite?

(1h 10m 10s)

Do we call it like, do we call it love?

(1h 10m 11s)

Like this is a normal part of the job.

(1h 10m 14s)

And I think just kind of collecting all this stuff together,

(1h 10m 18s)

painstaking work as it may be,

(1h 10m 19s)

like just saves everyone a ton of time.

(1h 10m 21s) Tim Chaten:

Yeah. So an inspiration for this app sounds like from personal need, you said this is something that you as a designer need these symbols and up until now it’s been pretty archaic as far as

(1h 10m 37s) Clarko:

Yeah, yeah, and in my career, I’ve been very fortunate to work on a lot of zero to one,

(1h 10m 44s)

like product design or product redesign from scratch, where you’re just like, all right,

(1h 10m 49s)

let’s reset assumptions.

(1h 10m 51s)

And so if you’re working on a much older app, meanings are fairly set.

(1h 10m 55s)

You don’t want to upset the apple cart by changing things needlessly under customers by changing icons or changing the meaning of things.

(1h 11m 3s)

But for new products, or like I said, a lot of what I do.

(1h 11m 7s)

Nowadays is prototyping, it’s all brand new, and it all has to be pretty quick.

(1h 11m 11s)

And so finding, finding the right icon or finding the right label for the icon fairly quickly is it’s a big need for me.

(1h 11m 19s)

And it always has been, but I, I also studied linguistics in school.

(1h 11m 24s)

And so that’s just part of what you do.

(1h 11m 27s)

Like if you become a professional linguist, it becomes your job to document the real world usage of language.

(1h 11m 33s)

That’s that’s the job.

(1h 11m 35s)

And so it’s funny to me that this is all colliding.

(1h 11m 37s)

And I’m like, I’m not sure if I’m going to be able to do this.

(1h 11m 40s)

But iconography is super hard.

(1h 11m 41s)

When I drawing icons is super hard.

(1h 11m 44s)

That’s literally semiotics as a job and on the design team, you know, button labels are usually called micro copy.

(1h 11m 52s)

And there usually isn’t a dedicated micro copywriter.

(1h 11m 55s)

Like it just kind of falls to the designer to write something that makes sense very often, unless you’re on a, at a big company.

(1h 12m 1s)

So, you know, a tool that surfaces, good icons, good microcopy gives you some

(1h 12m 7s)

That’s valuable to me.

(1h 12m 8s) Tim Chaten:

So, when you find the icon you want to use in SymbolSource, is it as easy as hitting Command-C to have that in your clipboard to paste it into wherever it needs to go, or…

Sharing the symbols from the app

(1h 12m 22s) Clarko:

Yep. Yeah. And in code, or if you’re working with engineers, you need to give them that unique identifier. And so you need to be able to copy that. And if you’re using something like Figma or Sketch, where you want to paste the icon itself into the thing that you’re drawing, you want to copy that. And I’ve got that as an alternate keyboard shortcut or,

(1h 12m 47s)

you know, in the context menu, because in a weird turn of events.

(1h 12m 52s)

Apple also publishes all of these icons in their SF font, the San Francisco Pro font family. They’re added to the font as characters, like in a far distant. Yeah. Yeah. They’re like non-standard emoji. They’re just in there, which it is limiting because you have to have the font to do that. But, you know, with print to PDF or, you know, just taking screenshots,

(1h 12m 59s) Tim Chaten:

Okay.

(1h 12m 59s)

Which, so you could paste it into pages and resize it as a text thing.

(1h 13m 8s)

Yeah.

(1h 13m 9s)

Okay.

(1h 13m 22s) Clarko:

that’s not a limitation on sharing that with the people who need to see it, like managers or whatever. But for tools like Sketch and Figma, you can just copy and paste that character into your design and it’s there and you can make it bold or you can make it bigger or smaller and you’re not losing any fidelity.

(1h 13m 39s) Tim Chaten:

Is this font built into all the Apple platforms or do you have to go out and download it?

(1h 13m 44s) Clarko:

To use it for copy and paste you have to download it, which is a bit of a bummer the system You when you’re using them in apps the system does it through a totally different mechanism?

(1h 13m 55s)

Through like through like some some API is for calling Images out of text because they you know for buttons and stuff. You’re typically using these as images. They’re not like yeah

(1h 13m 56s) Tim Chaten:

Okay, yeah.

(1h 13m 57s)

Yeah.

(1h 14m 3s)

Because it’s in code in that example.

(1h 14m 9s) Clarko:

So the font is an optional download and it’s pretty much only possible to install

(1h 14m 14s)

on a Mac, so that’s a big limiting factor. But…

(1h 14m 16s) Tim Chaten:

Oh, you cannot install this on an iOS device, there’s even, huh.

(1h 14m 23s) Clarko:

I, I have heard of someone trying to install it through like that, the old iTunes vector, I don’t even know if that’s possible anymore, but like with,

(1h 14m 29s) Tim Chaten:

Yeah.

(1h 14m 30s)

Huh.

(1h 14m 30s) Clarko:

without any, without any luck.

(1h 14m 35s) Tim Chaten:

That’s such a weird…

(1h 14m 37s)

How big is this font?

(1h 14m 41s) Clarko:

There are a few meg.

(1h 14m 43s)

I recently, I was recently using these fonts in a PDF and I used a few different weights of the font and you know, every weight of a font whether it’s bold or regular or thin is a whole separate font file.

(1h 14m 48s) Tim Chaten:

Mm-hmm.

(1h 14m 49s)

Right.

(1h 14m 50s)

Thank you.

(1h 14m 51s)

Thank you.

(1h 14m 52s)

Thank you.

(1h 14m 54s) Clarko:

And so that when then when I went to print the PDF of this document, it was like a five megabyte document or a five megabyte PDF because PDF embeds the font inside of the PDF file.

(1h 15m) Tim Chaten:

[laughs]

(1h 15m 6s) Clarko:

And so it’s like, oh, I can’t I can’t send someone a five megabyte PDF just because I

(1h 15m 11s)

the font. This is…

(1h 15m 12s) Tim Chaten:

Right. Yeah.

(1h 15m 14s)

In the Mac version of this app, is there going to be a way to walk people through installing these fonts on that Mac?

(1h 15m 22s)

Okay.

(1h 15m 22s) Clarko:

Yep, absolutely, because one of the, one of the weirder parts is if you’re running a beta operating system, those symbols are available to you like that year’s symbols, because they publish new ones every year, just, you know, just like emoji.

(1h 15m 34s) Tim Chaten:

Mm-hmm, right

(1h 15m 38s) Clarko:

So if you’re running a beta operating system, those symbols are available to you in code,

(1h 15m 42s)

but they’re not available to you to copy and paste from the font until they, until Apple releases a new version of the font.

(1h 15m 49s)

So there’s definitely on the Mac app I have a.

(1h 15m 52s)

Ton of different paths for just alerting you that it’s like, you’re not, you’re not going to have a good time.

(1h 15m 57s)

If you try and copy this particular symbol today, maybe wait a few months or.

(1h 16m) Tim Chaten:

Yeah. So apps use these, of course. What other kind of use cases and different projects are SF Symbols used in?

(1h 16m) Clarko:

Okay.

Where are these used?

(1h 16m 11s) Tim Chaten:

Like, are web pages starting to use these now? Or where else do we see these?

(1h 16m 16s) Clarko:

So Apple’s licensing is pretty restrictive.

(1h 16m 19s)

I haven’t seen, and I should read it more closely.

(1h 16m 24s)

I should take a magnifying glass to that license agreement to see if you’re allowed to subset the font.

(1h 16m 29s)

But all of the web tools that I’ve seen are very circumspect about like,

(1h 16m 38s)

about embedding the font in the webpage.

(1h 16m 41s)

They just kind of, they’ll put the glyph on the page and if you have the font installed,

(1h 16m 46s)

but if you don’t, then you just get, you know,

(1h 16m 48s)

the old question mark box.

(1h 16m 48s) Tim Chaten:

Gotcha. So web design.

(1h 16m 50s) Clarko:

Not so much, yeah.

(1h 16m 53s)

So very much just like things that are local to your computer and then Apple’s family of platforms for app design.

(1h 17m)

And there are certainly possibilities,

(1h 17m 3s)

like you are allowed to export and modify these.

(1h 17m 6s)

And some of them have very specific trademarks on them.

(1h 17m 11s)

Like you’re not allowed to export and modify the AirPlay symbol.

(1h 17m 14s)

That’s just AirPlay is an Apple trade.

(1h 17m 16s)

Mark, they don’t want you to do that.

(1h 17m 17s)

And you’re not allowed to export and modify the X-Box logo for obvious reasons.

(1h 17m 21s)

So Apple obviously has an agreement with Microsoft to include the X-Box logo, so long as they don’t let people mess with it.

(1h 17m 27s) Tim Chaten:

That’s fascinating the Xbox logo is a part of SF symbols that that’s a thing oh Duh the Sony and Nintendo have their logos in there as well probably Okay, yeah, cuz I know the joy cons are now supported finally Yeah

(1h 17m 27s) Clarko:

I think that’s, I think that’s because of, um, third-party controller support.

(1h 17m 36s)

Because I don’t know if I’ve seen the Nintendo logo, but Sony, yes.

(1h 17m 46s)

That’s good to know, but yeah, so it’s like they are ultimately images, and so they can ultimately be exported in any image format.

(1h 17m 55s)

But I’ve been holding off on that, obviously just to try and hit a 1.0 ship date, and because I really need to look at that license agreement to make sure I’m not gonna get in trouble for doing it.

(1h 18m 3s) Tim Chaten:

Yeah, yeah. So this app started life before Vision OS was a thing, correct?

Origins of the app

(1h 18m 15s) Clarko:

It did. Yeah, it actually started as a Mac OS app.

(1h 18m 15s) Tim Chaten:

Yeah. Okay, which makes sense with the font being there. Yeah.

(1h 18m 19s) Clarko:

Yeah.

(1h 18m 23s)

And, you know, Xcode, Figma, Sketch, all these tools that developers and designers are using every single day are on the Mac. And so that, for the longest time for me, was the primary interface.

(1h 18m 31s) Tim Chaten:

Right

(1h 18m 35s) Clarko:

And then because I spend so much time prototyping and sketching on my iPad, then I was like, "Well, I need an iPad app. Otherwise, you know, I’m just I just have to go and open my Mac to look up.

(1h 18m 42s) Tim Chaten:

Yeah, right.

(1h 18m 45s) Clarko:

So I added iPad support and then you know, if you make an iPad window small enough, it is essentially an iPhone app and might as well add iPhone support and so it just kind of snowballed from there and then when Vision OS was announced.

(1h 18m 53s) Tim Chaten:

Might as well Did the app Apple watch get might as well yet Yes, very smart yes, so What design considerations did you have to rethink?

(1h 19m 1s) Clarko:

I’m tempted by Apple Watch and TV OS, but I’m like at least smart enough to not jump on that for 1.0.

(1h 19m 15s)

[laughs]

Designing for Apple Vision Pro

(1h 19m 21s) Tim Chaten:

with Apple Vision Pro.

(1h 19m 23s)

So, what do you think?

(1h 19m 35s) Clarko:

Well, so we talked about how earlier like Vision OS apps are very, very similar.

(1h 19m 40s)

They’re definitely cut from the same cloth as iPad apps.

(1h 19m 43s)

And so thankfully not a bottom up redesign, like I didn’t have to like throw out a ton of stuff.

(1h 19m 48s)

But in some ways, you know, Vision OS is more like Mac OS, like we’re talking about buttons being too attention grabbing on iPad OS, because iPad OS comes with the assumption of running full screen.

(1h 20m 2s)

Like most apps are run full screen.

(1h 20m 5s)

And like stage manager is a very niche use case, but Mac OS and vision OS,

(1h 20m 11s)

you come with the assumption of multitasking.

(1h 20m 13s)

You are going to be one window of many.

(1h 20m 16s)

And there is such a thing as standing out for the wrong reasons in that environment.

(1h 20m 16s) Tim Chaten:

Yes Yeah

(1h 20m 21s) Clarko:

So I think there will be some, like some toning down of design from a lot of apps for vision OS.

(1h 20m 28s)

And I’ve done a little bit of that, like, you know, toolbar buttons that were hot pink.

(1h 20m 32s)

I’m like, oh, that’s–

(1h 20m 35s)

I’m like, oh, that’s jarring.

(1h 20m 36s)

And so, you know, I think if you look at a lot of the Vision OS apps in the keynote and on the website and stuff, like, everything is pretty monochrome because there’s just a lot of color in your environment that’s coming through.

(1h 20m 44s) Tim Chaten:

Mm-hmm.

(1h 20m 45s)

Yeah.

(1h 20m 46s)

Yeah.

(1h 20m 47s) Clarko:

And you need to contrast with that.

(1h 20m 50s)

So like, really, they’re just like, white is the contrast color.

(1h 20m 53s)

That’s– that is pretty much just what Vision OS is all about.

(1h 20m 56s)

The foreground is white because there’s no dark mode.

(1h 20m 58s)

It’s always just kind of like adaptive.

(1h 21m) Tim Chaten:

Yeah And we have not really seen from Apple’s material what?

How AVP became a primary target

(1h 21m 5s) Tim Chaten:

What using vision pro in the pitch black will be like?

(1h 21m 9s)

With those cameras and have all those perform and conveying your surroundings It does

(1h 21m 10s) Clarko:

Yeah, I mean, it has it has LIDAR, but I also just have no idea what LIDAR does in an environment.

(1h 21m 22s)

If there’s many vision pros all like shooting LIDAR are over the room.

(1h 21m 25s) Tim Chaten:

Right, yeah. Yes, right.

(1h 21m 27s) Clarko:

I just I don’t understand how any of this works under the hood because it’s not my area.

(1h 21m 34s)

But you know, besides like toning down your buttons and like trying to be a good multitasking citizen.

(1h 21m 40s)

There’s something, if I can embarrass myself as a developer, I was talking about how macOS was the primary interface.

(1h 21m 50s)

It’s where I expect most people to use the app.

(1h 21m 53s)

That means the iPad and then the iPhone are secondary and tertiary, and they were derivative interfaces, and simplified or at least made correct to their platform idiom, but not radically rethought because I’m one person and I’m trying to ship an app on three

(1h 21m 59s) Tim Chaten:

Yeah, sure.

(1h 22m 10s) Clarko:

platforms. Using Vision OS and seeing how much it has in common with iPad OS, it’s like,

(1h 22m 18s)

“Oh, okay. Now Mac OS is starting to be outnumbered.” And I personally consider Vision OS to be a primary productivity platform. I plan to use Xcode and Figma in Vision OS because I plan to just be using my Mac at the kitchen counter through Vision Pro.

(1h 22m 37s) Tim Chaten:

Yeah, the interesting use case is, so Xcode will run on the Mac and in VisionOS, you just get one window for that Mac with version 1 of this iteration, at least with version 1.0 of the software. So it highly encourages your Mac to be running a full screen app, in my mind,

(1h 22m 56s)

in this case, Xcode, and then SimpleSource being a different full screen app running in VisionOS.

(1h 23m 5s)

And I’d imagine copying.

(1h 23m 7s)

Between all the platforms will work as it does with Mac to iPad with the universal clipboard.

(1h 23m 12s)

So I’d imagine, yeah, we’ll be using some productivity apps within Vision OS to talk

(1h 23m 20s) Clarko:

Yeah.

(1h 23m 20s)

So it kind of like, to my mind, it makes vision OS a primary platform and iPad OS, because it has so much in common.

(1h 23m 27s)

Is this also a primary platform?

(1h 23m 27s) Tim Chaten:

Yeah.

(1h 23m 28s)

Yeah.

(1h 23m 29s)

Yeah.

(1h 23m 29s) Clarko:

And now it’s like, all right, is Mac OS of legacy platform now?

(1h 23m 30s) Tim Chaten:

Yeah.

(1h 23m 31s)

Yeah.

(1h 23m 32s)

Yeah.

(1h 23m 32s)

Yeah.

(1h 23m 32s) Clarko:

Am I, am I rethinking this entire app based on iPad as idiom?

(1h 23m 33s) Tim Chaten:

Yeah.

(1h 23m 34s)

Yeah.

(1h 23m 35s)

Yeah.

(1h 23m 36s)

Yeah.

(1h 23m 37s)

Yeah.

(1h 23m 37s) Clarko:

And I’m like, and I did, I, I just, I sat down and I was like, okay, I’ve made all these design decisions based on Mac OS idiom, and I’m going to throw home out because.

(1h 23m 50s)

Doing it this other way feels much better on iPad, and so it will also feel much better on Vision, and that’s what I want.

(1h 23m 56s)

And so I’m in the process of just redesigning huge chunks of the app just because of that like mental shift that I’ve made, and now that means adapting the Mac UI from the iPad UI instead of the other way around.

(1h 24m 10s) Tim Chaten:

Yeah Are the SF symbols available in any kind of three-dimensional?

3D Symbols?

(1h 24m 15s) Tim Chaten:

Format or have you considered some way of lifting?

(1h 24m 19s)

Lifting the symbols off the screen slightly to be like you know you know fake fake 3d 3d almost Okay, okay

(1h 24m 27s) Clarko:

They aren’t, um, and a lot of them would do poorly at that.

(1h 24m 33s)

Uh, like, so like I’ve, I’ve thought about it.

(1h 24m 36s)

Um, I’ve seen some, some efforts, I think.

(1h 24m 39s)

Steve Trout and Smith again, um, at like extruding shapes, like taking a 2d shape and making it pseudo 3d.

(1h 24m 41s) Tim Chaten:

Mm-hmm, yeah, yeah

(1h 24m 46s) Clarko:

Um, because vision OS apps, like if you go side on there, they’re,

(1h 24m 50s)

they’re not even paper thin, they are non-existent side on.

(1h 24m 54s)

And so there’s something about making.

(1h 24m 57s)

It a little 3D that’s interesting, but I am also like I’ve used plenty of font management apps over the years that like let you explore all the characters in a font.

(1h 25m 4s)

And then they’re like, what if we rendered it in brass?

(1h 25m 7s)

What if we made it?

(1h 25m 8s)

And I under I understand if you’re someone who who sells like door numbers for a living that you do actually want to see things extruded in brass because that’s what you’re selling.

(1h 25m 16s) Tim Chaten:

Right.

(1h 25m 17s)

Yeah, it’d be fun if, you know,

(1h 25m 18s) Clarko:

I’m just I’m not 100% sure that’s necessary for vision for for symbols.

(1h 25m 25s) Tim Chaten:

you integrate these symbols with like,

(1h 25m 27s)

I’m designing a VCR and I can lift the play button off the SymbolSource and drag it into my fake VCR I’m making.

(1h 25m 33s) Clarko:

Oh, so, I better not get in trouble for this.

(1h 25m 36s) Tim Chaten:

can.

(1h 25m 42s) Clarko:

Fun fact about Apple campus is they just use symbols everywhere.

(1h 25m 45s)

They’re like, they’re SF symbols on their trash cans and on their recycle bins and on their elevator buttons.

(1h 25m 50s) Tim Chaten:

Restrooms that imagine

(1h 25m 51s) Clarko:

They just like, yeah, they’re just like, why would we draw this again when we have all of these icons at our disposal?

(1h 25m 59s) Tim Chaten:

Yeah.

(1h 25m 59s) Clarko:

And you’re seeing that more and more in dub dub presentations as well.

(1h 26m) Tim Chaten:

Right.

(1h 26m 1s)

Yeah.

(1h 26m 3s) Clarko:

A lot of people giving presentations are like, man, I would have had to ask someone to draw me a thing, but now I’ll just dig up a totally new one.

(1h 26m 9s) Tim Chaten:

Yeah, definitely.

The size of visionOS

(1h 26m 11s) Tim Chaten:

Designing your app, you mentioned the iPad is kind of the best way to be designing now.

(1h 26m 17s)

If it feels good there, it’s going to feel good in Vision OS.

(1h 26m 20s)

Does it just feel roomier in Vision OS from what you’ve seen in WWDC videos?

(1h 26m 29s) Clarko:

Yeah, I mean, I think roominess feels relative to the space that you’re constrained in, like,

(1h 26m 35s) Tim Chaten:

• Yeah.

(1h 26m 35s) Clarko:

I don’t, I don’t open a lot of windows on my iPad, unless I’m like, unless I’m actively multitasking, because the screen just gets full really quickly, and it feels cramped.

(1h 26m 36s) Tim Chaten:

Yeah.

(1h 26m 47s) Clarko:

But if you’re doing what our like our multitasking friend did in the keynote, and you have four windows just kind of like arrayed around you, and like the music app is off to your left,

(1h 26m 56s)

It’s not even in your peripheral vision.

(1h 26m 58s)

You have to turn your head.

(1h 27m)

Cool.

(1h 27m)

You like most of your day, you’re not even looking at the music app, but it’s there and you can just turn your head at any time.

(1h 27m 5s)

It’s like, that is, that’s roomy.

(1h 27m 8s)

Cause you’re really just looking at what you want to look at without a lot of stuff, just in the periphery.

(1h 27m 14s) Tim Chaten:

As far as current functionality in your app, you mentioned some of what’s there.

Feature Complete?

(1h 27m 21s) Tim Chaten:

Is there anything you haven’t mentioned yet as far as what’s there?

(1h 27m 24s)

And as far as additional features, anything you hope to add for launch?

(1h 27m 29s)

Or is it just a matter of polishing up the current UI to be able to ship?

(1h 27m 35s) Clarko:

I mean, besides that major undertaking of redesigning the UI,

(1h 27m 39s) Tim Chaten:

Yeah, as far as that, that little thing, you know.

(1h 27m 39s) Clarko:

’cause this app is feature complete.

(1h 27m 41s) Tim Chaten:

[laughs]

(1h 27m 43s) Clarko:

It’s been on test flight for a while.

(1h 27m 44s)

Like, if I was smarter, I would just ship it.

(1h 27m 47s)

But I am instead like undertaking this major renovation.

(1h 27m 52s)

But there’s functionality built in that is very much geared towards developers,

(1h 27m 59s)

because that is the target market.

(1h 28m 1s)

And it’s, you know, if you see a symbol

(1h 28m 5s)

that you like it and you want to use it,

(1h 28m 6s)

but your app is targeting iOS 14 and greater,

(1h 28m 10s) Tim Chaten:

Yeah, oh, right, definitely.

(1h 28m 10s) Clarko:

and this symbol was published in iOS 15,

(1h 28m 13s)

you need to know about that.

(1h 28m 14s)

So I have, like, right now I show the date of release and let you also look at, you know,

(1h 28m 21s)

what specific, what that corresponds to for different platforms,

(1h 28m 24s)

because a symbol that came out in,

(1h 28m 26s)

technically in September of 2023,

(1h 28m 30s)

it isn’t, oh wait, it is out yet for Sonoma, nevermind.

(1h 28m 33s) Tim Chaten:

Yeah, right.

(1h 28m 35s) Clarko:

September of 2022, symbols that came out for iOS 16 didn’t hit macOS for another month.

(1h 28m 43s)

And so there’s like, there’s that entire mapping of releases to OS versions to dates that is like weird and interesting.

(1h 28m 50s) Tim Chaten:

Right, is there going to be a filter to like filter out “I just want stuff from this point forwards”?

(1h 28m 51s) Clarko:

Yeah, and so I have, it’s funny,

(1h 28m 59s)

there’s a whole treasure trove of features that I’ve just kind of like hidden

(1h 29m 5s)

and I want to ship instead.

(1h 29m 6s) Tim Chaten:

Mm-hmm. Yeah.

(1h 29m 6s) Clarko:

It’s like, as soon as 1.0 ships,

(1h 29m 9s)

then I can unhide those features and make sure they’re good and then ship a 1.1,

(1h 29m 12s)

and that’ll almost certainly be before Vision Pro.

(1h 29m 15s)

But things like that version filter,

(1h 29m 17s)

where you’re like, I am targeting iOS 13,

(1h 29m 21s)

only show me symbols from iOS 13 and earlier.

(1h 29m 24s)

Like, that’s critical to like anybody working on an app that’s been around for more than five minutes.

(1h 29m 29s)

And things like previewing it in your company’s colors,

(1h 29m 30s) Tim Chaten:

For Mac, yeah.

(1h 29m 35s) Clarko:

if you’ve worked on an app and your app is tinted to the company colors,

(1h 29m 39s)

that’s what you’re looking at all day.

(1h 29m 40s)

And you just kind of want to see it previewed that way.

(1h 29m 42s) Tim Chaten:

Yeah.

(1h 29m 43s) Clarko:

Things like that, where it’s just like little quality of life features for designers and developers.

(1h 29m 43s) Tim Chaten:

As far as input in VisionOS, I’d imagine you have to design this app with the expectation that they may not even have a keyboard or mouse in front of them, entirely hand in eye.

Input in visionOS

(1h 29m 58s) Tim Chaten:

But also, most of them probably will also have a keyboard and mouse because they’re working in Xcode alongside this.

(1h 30m 5s)

How do you design with that in mind of input versatility?

(1h 30m 11s) Clarko:

That feels a lot like iPadOS, really.

(1h 30m 12s) Tim Chaten:

Right?

(1h 30m 14s) Clarko:

It’s interesting because a lot of apps, mine included,

(1h 30m 19s)

the input, especially text input, is really limited.

(1h 30m 24s)

Like, it’s a search field.

(1h 30m 24s) Tim Chaten:

Yeah, right.

(1h 30m 26s) Clarko:

There are some other fields for text input,

(1h 30m 28s)

but this is not an app for composing text the way that Xcode is,

(1h 30m 32s)

or IARider, or any of these apps where you would expect to use a hardware keyboard.

(1h 30m 37s)

Because I like I really use my hardware keyboard

(1h 30m 41s)

a lot because I’m either writing text or I’m writing code.

(1h 30m 44s)

And going back to our conversation about gestures earlier,

(1h 30m 49s)

design tools use keyboard very heavily, even though you think of like designing is a very mouse heavy process or or stylus heavy process.

(1h 31m)

The number of keyboard modifiers, the number of keyboard shortcuts that you have to use just to get any work done at all in design tools is kind of horrifying.

(1h 31m 9s)

There’s an old joke about the photo.

(1h 31m 11s)

Photoshop save for web claw, you have to like contort your hand into a horrible little claw in order to press the six buttons in order to save for web.

(1h 31m 21s)

And you know, everybody gets RSI eventually.

(1h 31m 24s)

And so I’m like, actually, like I said about ergonomics earlier, very excited about this platform that it could solve a lot of that.

(1h 31m 32s)

But I’m certainly going to bring my keyboard to it to start with, just because that’s,

(1h 31m 38s)

that’s the habit I formed.

(1h 31m 39s)

I’m a keyboard person.

(1h 31m 41s)

And maybe I can switch to a split keyboard or like, you know, can you buy trousers with keyboards in the legs or something?

(1h 31m 46s)

There’s got to be something there that makes it a little better.

(1h 31m 50s)

But I’m really curious how text input will evolve over time.

(1h 31m 56s)

Because programming is heavily keyboard bound, as I said.

(1h 31m 59s)

But like, can you have like a multi-finger mid-air swipe keyboard?

(1h 32m 4s)

Like if you’re using all 10 fingers, can you just type out a word just by doing,

(1h 32m 6s) Tim Chaten:

Right.

(1h 32m 7s)

Yeah.

(1h 32m 9s) Clarko:

by casting a magic spell.

(1h 32m 11s)

Um, can I, can I dictate at whispering volume or do I have to be announced?

(1h 32m 15s)

Or do I have to be enunciating loudly the way that I feel like I have to do with Siri, it’s, it seems like there’s so much potential here, like, and earlier we’re talking about gestures, like, yeah, I want to be able to just like, I don’t know,

(1h 32m 28s)

like flick my thumb and have a backspace or an undo.

(1h 32m 32s)

Like I, there’s so much potential for fluency that doesn’t rely on a keyboard.

(1h 32m 37s)

And I’m just like, I’m waiting for it.

(1h 32m 38s)

’cause it’s like they haven’t shown it.

(1h 32m 41s)

And we know what the standard gestures are ’cause they published them.

(1h 32m 44s)

And we’re just, we’re waiting on like that next level,

(1h 32m 47s)

that next level,

(1h 32m 48s)

which could honestly be five years in the making.

Multiple Windows per App?

(1h 32m 53s) Tim Chaten:

So iPadOS is very constrained to a single window. You know, you can do multiple windows, but it’s really not, not, not, doesn’t, you know, shine there. VisionOS, on the other hand,

(1h 33m 5s)

you got all over the world, multiple windows, these ornamental windows for your apps.

(1h 33m 10s)

What are your thoughts on having, you know, pop out windows to your app? Does it make sense in this case, or does it make best sense?

(1h 33m 23s) Clarko:

I mean, I think this platform is geared towards power users, and I think, like, maybe it’s a gimmick, maybe it’s something to do with eye tracking, but the ornaments and stuff that are in different depth planes, there’s something there that probably has a good explanation and they’re not telling us.

(1h 33m 42s)

But I think that it’s nice and it’s interesting that they can say, "Oh, we’re not bound by the rectangle anymore.

(1h 33m 49s)

If it doesn’t have to be in the rectangle, then don’t put it in the rectangle.

(1h 33m 53s)

But I think Vision OS is as suited to multi-window stuff as macOS ever was,

(1h 33m 59s) Tim Chaten:

Yeah. Yes. Mm-hmm.

(1h 33m 59s) Clarko:

which makes me happy because it is a power user platform.

(1h 34m 1s)

But even saying that, like there’s been a clear trend on macOS away from floating panes into like contained sidebars and things like, I don’t even know if there were such things as photo viewers back in the day, but you probably just looked at your finder window and clicked on a photo in finder and opened it in a new window

(1h 34m 22s) Tim Chaten:

Yeah. Occasionally you’d hit the spacebar to play it as quick look.

(1h 34m 23s) Clarko:

and now we have photos.

(1h 34m 24s)

Yeah.

(1h 34m 28s)

And QuickLook is like, relatively speaking, young, but things like, like the photos

(1h 34m 33s) Tim Chaten:

I was just thinking cover flow as a very horizontal window floating above your entire space in a 3D way could be a lot of fun.

(1h 34m 48s) Clarko:

Oh, yeah.

(1h 34m 49s)

Like, I mean, we said, you know, they are, they’re obviously being very conservative about what they ship in 1.0, but like the, the space management that they have to be playing with to ship like three or four years from now has got to be something really interesting because yeah, I want to save a certain number of windows in a certain configuration and pull them out of my hat.

(1h 34m 57s) Tim Chaten:

Yeah.

(1h 35m 9s)

Yeah, yes, yeah

(1h 35m 9s) Clarko:

Like I essentially want spaces like the Mac OS version of spaces.

(1h 35m 13s)

I want it in virtual space.

(1h 35m 15s)

But for now, I’ll be happy to work from home.

(1h 35m 18s)

my living room and just like leave some windows over by the couch and leave a different set of windows over by the kitchen table and I’ll just physically move if I need to use different apps.

(1h 35m 26s)

But yeah, there’s got to be something interesting they’re working on for for space management.

(1h 35m 32s) Tim Chaten:

Yeah, I can’t imagine developers making use of these very narrow, wide app windows.

(1h 35m 39s)

That just would not make sense in any other platform.

(1h 35m 41s)

I think that could be pretty interesting.

(1h 35m 44s) Clarko:

Hmm. I think, I think, yeah, multi-window from a single apps perspective is, is welcome.

(1h 35m 51s)

Even though iPadOS clearly dissuades you from it. Like, it’s possible on iPadOS, but, but it’s, it’s crowded. Yeah. And VisionOS design guidelines actually warn against doing too much with, with too many windows because it can apparently feel crowded. And I don’t think designers who ever.

(1h 35m 55s) Tim Chaten:

Yes, it’s possible, but you don’t see floating tool palettes very often.

(1h 36m 7s)

Okay.

(1h 36m 8s)

Yeah.

(1h 36m 14s) Clarko:

Like put a great deal of effort into making sure things don’t feel claustrophobic before, but now that’s like, that’s a legitimate concern that your user interface could actually cause people distress.

(1h 36m 25s)

So that’s, that’s on the designer’s head.

(1h 36m 28s) Tim Chaten:

Yeah, you have to look at all the phobias, make sure your app doesn’t say the spider phobia or whatever, the whole phobia.

(1h 36m 34s) Clarko:

Mm-hmm, yeah, the too many small holes.

(1h 36m 35s) Tim Chaten:

Yeah.

(1h 36m 36s) Clarko:

But yeah, I think, I think as a user, the freedom to open multiple windows is more than welcome.

(1h 36m 42s)

Um, like I certainly have already put like an open a new window button in my context windows, because, because why not?

(1h 36m 49s)

And I, I don’t use that context.

(1h 36m 50s) Tim Chaten:

Right.

(1h 36m 51s) Clarko:

I don’t use that button on iPadOS.

(1h 36m 53s)

I’ve never felt the need, but EnvisionOS, if I can open something in a little window and then throw it onto the other side of the room for safekeeping, then I will.

(1h 37m 1s) Tim Chaten:

Yeah, that’d be kind of funny if, you know, you’re filing cabinets in the real world,

Filing cabinets and BT accessories

(1h 37m 9s) Tim Chaten:

you just open one up that’s empty, you just put a window in there for safekeeping for later that you can pull out.

(1h 37m 10s) Clarko:

[laughing]

(1h 37m 15s) Tim Chaten:

I was just thinking about that.

(1h 37m 18s) Clarko:

Where you like program the action button on your phone to like switch between different windows in your Vision OS.

(1h 37m 25s) Tim Chaten:

Yeah. Right. Yeah.

(1h 37m 27s)

There should totally be some kind of action button integration.

(1h 37m 36s) Clarko:

I mean, part of me wants to, but knows that this is a zero margin business that you really struggle to make any money in.

(1h 37m 44s)

But I want to make Bluetooth accessories where it’s just like, you know how you can buy stomp boxes that add effects to your guitar output?

(1h 37m 50s) Tim Chaten:

Yeah.

(1h 37m 52s) Clarko:

I just want stomp boxes all around.

(1h 37m 55s)

When I step on it, it just talks to my device and triggers something.

(1h 37m 59s) Tim Chaten:

Yeah, that’d be great.

(1h 38m) Clarko:

And the closest we’ve got is like weird HomeKit buttons that like go through my Ikea hub to

(1h 38m 6s) Tim Chaten:

Right, if they make Bluetooth pedals that talk to iPad, we’re going to change…

(1h 38m 6s) Clarko:

to do weird stuff, but.

(1h 38m 15s)

Yeah, I just want buttons everywhere.

(1h 38m 17s) Tim Chaten:

One thing I was thinking of, collections, like as you’re working on different projects,

Collections

(1h 38m 23s) Tim Chaten:

probably not on version 1, but version 2, I’d imagine favoriting certain things for certain collections of icons you’re working with for different projects.

(1h 38m 33s)

Yeah, yeah.

(1h 38m 33s) Clarko:

Yep. Absolutely. That’s, it’s one of those things where it’s like, okay, that’s a great feature, but I want it, I would want it to sync, which means I need to like, look into cloud kit. So like, let’s just put that on the back burner. It’s one of those things where it’s like, you look at the scope balloon, and then you’re like, okay, not today.

(1h 38m 45s) Tim Chaten:

Yeah. Yeah. Yes. Yes.

Anything else about your app?

(1h 38m 50s) Tim Chaten:

Anything else about SymbolSource before we move on?

(1h 38m 54s) Clarko:

No, I mean, I’m, I’m kicking myself for this, like, major renovation that I’m doing it and I should just ship it already. But no, it’s I’m happy.

(1h 39m 3s)

I’m happy to like, get it right. It is one of those things, which is definitely a curse, because I have a lot of friends who have a much smarter view of shipping software where they’re just like, all right, let’s, let’s build a concept and ship it and if it has any traction at all, then, then it’s got legs and let’s work on it more.

(1h 39m 21s)

And I’m just like, I’m just like, no, I already know that I want this. And I already prototyped it for myself. And I know that it has legs. And I know it has legs for myself. So I’m just gonna sit here and polish it for a good long time.

(1h 39m 28s) Tim Chaten:

Right.

(1h 39m 29s)

[laughs]

(1h 39m 33s) Clarko:

Before I put it out in the world. And maybe that’s a side effect of working in so many hardware companies where it’s like, the development time for hardware is so long that you are given the liberty of, of years to work on the software that goes with that hardware.

(1h 39m 50s)

And you know that people are mostly not updating the software on those hardware devices. So you you want to get it right out of the box. So it’s, there’s some, some brain damage I’ve taken from working in hardware startups that is now.

(1h 39m 58s) Tim Chaten:

Right, yeah. So we’ve already chatted a bit about, you know, cat apps and other things like that. Any Vision OS apps you hope get created by either yourself or someone else one day soon?

(1h 40m 3s) Clarko:

now leaking into my software life.

What visionOS apps do you hope get made?

(1h 40m 17s) Clarko:

Yeah, actually, if anyone listening has millions of dollars, please hit me up and I’ll, I’ll hire a team to work on this.

(1h 40m 24s)

Because this is this is not for me.

(1h 40m 25s)

I think it would just be so cool that you know how a lot of movies, almost all action movies nowadays are just shot entirely on green screen with the little tracking balls.

(1h 40m 34s)

And then they’re composited with background plates.

(1h 40m 37s)

And in the old days, like in Star Wars original trilogy, those backgrounds were all hand painted.

(1h 40m 42s)

But now they’re 3d models.

(1h 40m 45s)

And I assume that’s it.

(1h 40m 47s)

The job of a set designer in this world is just more difficult than it needs to be.

(1h 40m 50s) Tim Chaten:

Yeah.

(1h 40m 51s) Clarko:

This is like, you’re not just a set designer, you’re a 3D artist.

(1h 40m 56s)

Or those are now two totally different jobs.

(1h 40m 58s)

There are set designers who literally go thrifting for cool old objects and put them on a shelf for when they need a physical set.

(1h 41m 5s)

And there are 3D artists who do something entirely different and they don’t go thrifting or maybe they do just for inspiration.

(1h 41m 10s)

But I think an AR tool for virtual set design, or just virtual interior design, would be

(1h 41m 17s)

incredibly fun.

(1h 41m 18s)

Just having this Mary Poppins bag full of furniture and books and art and tchotchkes to just put in a virtual space and see how it feels.

(1h 41m 27s)

I think that would be so fun.

(1h 41m 29s) Tim Chaten:

Yeah.

(1h 41m 29s) Clarko:

And I’m a habitual furniture rearranger, so maybe that’s coming into it here, but rearranging furniture with your eyes and repainting with the snap of a finger and not having to drill holes to hang a new piece of art.

(1h 41m 41s) Tim Chaten:

Yes.

(1h 41m 44s) Clarko:

I spent the last few weekends changing out light fixtures and.

(1h 41m 47s)

And stuff around the house, so it’s I’m feeling it right now.

(1h 41m 48s) Tim Chaten:

Yeah.

(1h 41m 49s)

Yeah.

(1h 41m 50s)

Yeah.

(1h 41m 52s) Clarko:

Moving furniture and standing on ladders.

(1h 41m 54s)

And it’s like it’s it sounds fun to me and it also sounds really useful to a very specific industry.

(1h 42m)

And so I think that would be incredible to work on.

(1h 42m 2s) Tim Chaten:

Yeah, that’d be super cool and it’d be amazing if you know integrate with Final Cut Pro and the other Video editors and whatever they used to compile this stuff Yes Yeah, that’d be cool Yeah, a lot of sets are indoors some stuff is outdoors yet the I’m curious so that as we’ve talked about earlier how how outdoors will behave Yeah Any final thoughts regarding spatial computing?

(1h 42m 9s) Clarko:

Mm-hmm and the gestures the infinite gestures you could invent for that workflow

Final thoughts?

(1h 42m 32s) Tim Chaten:

before we start to wrap it up.

(1h 42m 34s)

Or before we wrap it up, yeah.

(1h 42m 37s) Clarko:

We talked a bit earlier about like 3d objects as as things that you might want to subscribe to like a feed of I’m just like ah Yeah, I just I know that they they demoed that Dinosaur thing but it’s like there are so many more things and places in the world that are not safe for me to go or Not safe for me to be near that. I want to and it’s like alright if nature documentaries can all be shot in 3d

(1h 43m 2s)

I’ll be a happy camper or if like some 3d artists just produce like a life-size

(1h 43m 7s)

grizzly bear model that I can like stand near because I’ve only ever stood,

(1h 43m 11s)

you know, 60 feet from a grizzly bear at a zoo and I’m not getting,

(1h 43m 13s) Tim Chaten:

Yes.

(1h 43m 15s) Clarko:

not getting any closer. And I’ve never stood on the edge of a volcano.

(1h 43m 18s)

There are all these things that I will never do that I would love to see up close. And I think there’s like,

(1h 43m 24s)

that’s another one of those ideas that is gated entirely by if people can make 3d objects for it.

(1h 43m 29s)

But I just,

(1h 43m 32s)

There is so much to see in a virtual world because of.

(1h 43m 37s)

You know, the constraints of the real world that don’t let us poke at blue whales in the flesh.

(1h 43m 42s) Tim Chaten:

Yes. The Horizon, Call of the Mountain, it really, you start the game in this boat ride with these monster machines jumping over you and it, the sense of immersion is there. It really is. I can’t wait to experience stuff like that in Apple Vision Pro and National Geographic being owned by Disney. Perhaps they will.

(1h 43m 58s) Clarko:

[laughs]

(1h 44m 6s)

Mm-hmm Yeah, now you’re now you’re encouraging me to buy a psvr – I I Held I held off because the only game I was really interested in playing was moss and I’ve already played moss I just wanted to play it in higher res

(1h 44m 12s) Tim Chaten:

You totally should. It has eye tracking so you can play around with how that all works in some of the games.

(1h 44m 26s)

You mentioned you are not a racer, but Gran Turismo,

(1h 44m 29s)

you will want to then buy a racing wheel,

(1h 44m 30s)

which is an incredible experience in that as well.

(1h 44m 33s) Clarko:

Okay, expensive hobbies then.

(1h 44m 34s) Tim Chaten:

Yeah. (laughs)

(1h 44m 36s)

Yeah.

(1h 44m 37s)

Yes, yes.

(1h 44m 38s)

It’s a rabbit hole.

(1h 44m 39s)

That’s a fun one though.

(1h 44m 41s)

Well, thank you so much for your time.

(1h 44m 42s)

This has been a really fun chat.

(1h 44m 44s)

I really enjoyed learning more about Vision OS.

(1h 44m 46s) Clarko:

Thank you so much for having me.

(1h 44m 47s) Tim Chaten:

Yeah, absolutely.

Where can people find you?

(1h 44m 49s) Tim Chaten:

Where can people find you on the interwebs and when your app launches,

(1h 44m 54s) Clarko:

I am @clarko on mastodon.social and on too many other social medias to bother with, but I’m really only paying attention to Mastodon.

(1h 45m 5s)

It feels like the Twitter of 2012, so it’s a safe place for dorks.

(1h 45m 12s) Tim Chaten:

Yes.

(1h 45m 13s)

Excellent.

(1h 45m 13s) Clarko:

And that is where I’ll be posting about Symbolsaurus and any future projects.

(1h 45m 14s) Tim Chaten:

Well, thank you so much.

(1h 45m 17s)

Really appreciate your time today.

(1h 45m 20s)

Made with Transcriptionist