Back at it
-
I figured why not use RealityKit for the material previews, so now they are actual spheres.
Miraculously, it all still works — the Metal viewport, the Metal immersive scene, and the RealityKit UI elements, but it's very clear the Vision Pro (M2) doesn't have much headroom to build an actual app around this stuff
The raytracer is, for now, a no-go on visionOS. It's possible I could throttle it and stay within visionOS' systemwide render budget. But it's probably worth improving the RT performance a bunch on its own first before I come back and try it here. I might run out of steam on this prototype before then.
This entire app project is still in my 'Temp' folder, where throwaway projects live
-
The raytracer is, for now, a no-go on visionOS. It's possible I could throttle it and stay within visionOS' systemwide render budget. But it's probably worth improving the RT performance a bunch on its own first before I come back and try it here. I might run out of steam on this prototype before then.
This entire app project is still in my 'Temp' folder, where throwaway projects live
This project, which runs on iPhone, iPad, Mac, and Vision Pro (with Immersive Space), is now 16.5K lines of code
-
The raytracer is, for now, a no-go on visionOS. It's possible I could throttle it and stay within visionOS' systemwide render budget. But it's probably worth improving the RT performance a bunch on its own first before I come back and try it here. I might run out of steam on this prototype before then.
This entire app project is still in my 'Temp' folder, where throwaway projects live
@stroughtonsmith “This entire app project is still in my 'Temp' folder, where throwaway projects live”
-
Pretty much everything I've worked on with Codex up to now has been stuff I could have built myself, within my area of expertise (or learnable), it just would have taken weeks or months.
This 3D scene app is something I never would have been able to build myself. I would have needed a team of rendering experts with domain-specific knowledge and human-years of research
@stroughtonsmith interestingly for me it is the reverse: a basic modeler with a basic built in ray tracer would take me maybe a day or two to build on my own with an imgui based UI.
But all of the native iOS/iPadOS/visionOS stuff would have required me to have dedicated Apple frameworks experts help out on. My experience with writing Apple native apps is super limited. But with Codex, I’ve now made several nice custom, just-for-me native apps to replace things that were just scripts before.
-
@stroughtonsmith interestingly for me it is the reverse: a basic modeler with a basic built in ray tracer would take me maybe a day or two to build on my own with an imgui based UI.
But all of the native iOS/iPadOS/visionOS stuff would have required me to have dedicated Apple frameworks experts help out on. My experience with writing Apple native apps is super limited. But with Codex, I’ve now made several nice custom, just-for-me native apps to replace things that were just scripts before.
@yiningkarlli you might be a special case, all things considered
-
@jlaase I did blog about it, and it has one prompt session transcript: https://highcaffeinecontent.com/blog/20260301-A-Month-With-OpenAIs-Codex
@stroughtonsmith@mastodon.sociadang, I guess I missed it. Thank you for pointing it out for me!
-
@yiningkarlli you might be a special case, all things considered
@stroughtonsmith Haha, that's fair. You might be too though in Apple stuff; not many other developers reverse engineered all of Catalyst out of thin air before they even announced Catalyst.
Anyhow all of this is very exciting/cool. I have a sort of researchy project that I was previously planning on just making into your usual kind of crappy janky graphics programmer demo app, but now I'm more ambitious and want to make the whole thing a really nice native Mac app with some help from Codex.
-
The raytracer is, for now, a no-go on visionOS. It's possible I could throttle it and stay within visionOS' systemwide render budget. But it's probably worth improving the RT performance a bunch on its own first before I come back and try it here. I might run out of steam on this prototype before then.
This entire app project is still in my 'Temp' folder, where throwaway projects live
@stroughtonsmith it’s really impressive and beautiful how this one is coming together.
-
So now that visionOS 26 lets you spawn immersive scenes from UIKit apps, I had Codex implement me an immersive scene using Metal and CompositorServices that mirrors the in-window viewport and lets you live in your scene
It's real frickin cool.
The raytracer might be off limits for visionOS, but there's a lot of interesting stuff to do in other areas
@stroughtonsmith This is getting scary
-
This project, which runs on iPhone, iPad, Mac, and Vision Pro (with Immersive Space), is now 16.5K lines of code
I thought it was finally time to add vertex editing and subdivision. Now it's a 3D modeling tool and not just a raytracer
-
I thought it was finally time to add vertex editing and subdivision. Now it's a 3D modeling tool and not just a raytracer
@stroughtonsmith Boolean solid operations next?
-
@stroughtonsmith Boolean solid operations next?
@juandesant good idea
-
I thought it was finally time to add vertex editing and subdivision. Now it's a 3D modeling tool and not just a raytracer
Some more things to show off here on this iPad mini 6!
• Longpress band gesture
• Multi-select
• Vertex editing
• Subdividing
• My 'generate a Cornell Box' button
• (And the raytracer, of course)There is a lot of really neat stuff in this app. Still using Codex 5.3 Medium, still haven't touched a line of code myself
-
Some more things to show off here on this iPad mini 6!
• Longpress band gesture
• Multi-select
• Vertex editing
• Subdividing
• My 'generate a Cornell Box' button
• (And the raytracer, of course)There is a lot of really neat stuff in this app. Still using Codex 5.3 Medium, still haven't touched a line of code myself
All of this still works great on iPhone too
-
All of this still works great on iPhone too
The touch gestures all work on visionOS too, but on all platforms it has keyboard and mouse support for all your precise selection and modifier key needs
-
The touch gestures all work on visionOS too, but on all platforms it has keyboard and mouse support for all your precise selection and modifier key needs
@stroughtonsmith It’s crazy to see how something like this can evolve. Do you find it weirdly addictive to see what the GPUs will cook up next?
-
@stroughtonsmith It’s crazy to see how something like this can evolve. Do you find it weirdly addictive to see what the GPUs will cook up next?
@brndnsh it's just a string of dopamine hits
-
The touch gestures all work on visionOS too, but on all platforms it has keyboard and mouse support for all your precise selection and modifier key needs
@stroughtonsmith “tsWork ‘26”
-
@brndnsh it's just a string of dopamine hits
@stroughtonsmith So many “one more prompt” moments…
-
The touch gestures all work on visionOS too, but on all platforms it has keyboard and mouse support for all your precise selection and modifier key needs
@stroughtonsmith out of curiosity. Do you use Codex from inside Xcode or do you use the CLI?
My results in generating functional iOS projects with the CLI have been very mixed
