Oculus Developer Blog

So far Meta has published a whole bunch of spatial APIs including scene detection, spatial anchors, plane detection etc. [0] They are far ahead of Apple in many of these respects, but what they haven't had is a device that made this attractive to actually develop for in any meaningful way. Quest Pro lacks a depth sensor and has too small a market presence to attract a lot of developers, while Quest 2 has ugly, black and white low resolution pass through. So Quest 3 will be the first spatial computing device the world has ever had access to that is available at mass market price (<$500) with depth sensor and high quality pass through. It will finally be worthwhile for developers to build mixed reality apps targeted to regular consumers. https://developer.oculus.com/documentation/unity/unity-spatial-anchors…

You can sign up as a developer to create software for Oculus devices https://developer.oculus.com/ and there is an Oculus Quest store https://www.oculus.com/experiences/quest/ If you developed your own "metaverse" you could list in the Quest store but of course Meta can push you around there the same way Apple can push you around in its App Store. Meta definitely has a vertical integration strategy, I think Horizon Worlds (or Horizon Worlds + Facebook) is what they have been after for a long time. If the hardware was there in 2014 (when they bought Oculus) I think they would have deployed a service on whatever hardware was available -- but given that the hardware needed serious investment they figured they could lead that revolution.

Increased latency due to using complex, deep learning based, entirely computer vision driven hand tracking vs. tracked controllers which partially use accelerometer data to correct drift and account for latency, and use a much faster classic computer vision algorithm for position tracking (https://developer.oculus.com/blog/tracking-technology-explained-led-ma…

This best practices guide for the Oculus was helpful for me: https://developer.oculus.com/documentation/intro-vr/latest/concepts/bp… It covers a lot of the concepts and what actions the developer can take to minimize (or exacerbate) problems with VR.

On an unrelated note, in 2021 Carmack and Meta announced [1] unlocking the Oculus Go bootloader, explicitly mentioning that it "opens up the ability to repurpose the hardware for more things today, and means that a randomly discovered shrink wrapped headset twenty years from now will be able to update to the final software version" [2]. I thought that was great, wanted to hack [3] on it a bit, so I went ahead and bought an Oculus Go on the second hand market. Turns out that the quote isn't true at all [4]. Updating the firmware requires enabling developer mode [5]. Enabling developer mode requires activating the Oculus Go with a Meta or Oculus account [6]. I don't have or want to have a Meta-related account, and in twenty years the online activation process likely won't work at all, making Carmack's shrink-wrapped headset useless without a fight. I'm sure the point is moot in twenty years because you'll just ask Skynet to write you an Android root exploit, but right know my second hand Oculus Go is surely acting like a brick in my drawer. [1] https://www.theverge.com/2021/9/27/22696030/oculus-go-root-access-unlo… [2] https://xcancel.com/ID_AA_Carmack/status/1441496416463904768 [3] https://github.com/AndroidDevice-Porting/android_device_oculus_pacific… [4] Although at first I couldn't believe this and assumed I must be doing something wrong instead. I'm still not ruling out that possibility; I would very much like to hear a…

My recommendation is to do the equivalent of FRC in your shaders. I use one of the dither functions from https://developer.oculus.com/blog/tech-note-shader-snippets-for-effici… to break up banding in my geometry rasterization shaders that I use for rendering all my UI elements. It looks good in static images and fantastic in motion since the dithering pattern shifts every frame. One thing to watch out for is that if you're alpha blending, you need to be fairly cautious about how you dither and multiply. Premultiplied alpha does not get along with dithering, so the output from your shaders needs to be dithered but NOT premultiplied, with premultiplication performed by the GPU's ROPs (they typically seem to operate above 8 bits of precision.) If you don't do this you will get really nasty banding on alpha fades even though you dither. Also, counter-intuitively, you may not want to dither the alpha channel (test it yourself, ymmv) When dealing with sRGB and linear space it can also be important whether you dither before or after the color-space conversion.

If you're good with computer graphics and programming, you're halfway done for doing your first demos / experiments :) If you have an OculusRift, I think that's probably the easier start. Not because "mobile" is harder, but because you can focus more on doing stuff instead of optimizing / cutting stuff because mobile devies still do not have so much computational power. Unity3D / Unreal engine are really easy to set up for VR. So easy to set-up that most people lose sight of VR. I HIGHLY suggest you to give a read here https://developer.oculus.com/documentation/intro-vr/latest/concepts/bp… and http://static.oculusvr.com/sdk-downloads/documents/OculusBestPractices… You may think this is simple, everyone do it but... no. There are so many useful hints about how to reduce sickness and produce a better product, and so easy to find, that is mandatory to know. This will get you started pretty well :)

OpenXR Mobile SDK is the native development option for the Quest devices: https://developer.oculus.com/documentation/native/android/mobile-intro…

Meta does a ton of ongoing work to optimize the stack in the headset. I was just reading about Application SpaceWarp (https://developer.oculus.com/blog/introducing-application-spacewarp/) earlier today and it's impressive.

Compare this with the Quest devkit: https://developer.oculus.com/quest/

The camera is only used to track the controller, the actual 3D data is pulled from the game files. The Quest API does let you layer elements onto the passthrough view[0], so it could work, the game would just hover in front of you instead of tracking the controller. [0] https://developer.oculus.com/documentation/unity/unity-passthrough/

I vividly recall Carmack describing some of the lengths they had to go to to get VR respond to body movements. IIRC it was crucial to get the scene to move around exactly when your head moves around, or else your body would think you’re poisoned/dying or something. The PC would be polling head position and rendering at like 90fps and even that was too much latency so they had this insane hack where the rendered frame was sent down the wire to the headset, and the headset itself had some onboard processor that would skew and shift the frame according to the last data from the accelerometer literally right before putting the frame on screen. https://developer.oculus.com/documentation/native/android/mobile-timew…

Again, it does happen on the headset, but that's only necessary in situations where there's not enough bandwidth to send an uncompressed video stream. See Oculus Link over USB, for example: https://developer.oculus.com/blog/how-does-oculus-link-work-the-archit… The diagram there shows Timewarp running on both the PC and the headset side. In headsets like the Rift or Valve Index that send video streams over HDMI or DisplayPort that extra step isn't necessary; everything happens on the PC's GPU.

Passthrough can be overlaid with synthetic elements. I don't get what you're on about. https://developer.oculus.com/blog/mixed-reality-with-passthrough/?loca… I use this stuff and develop apps for it.

For the arms you could use Meta's new Movement SDK: https://developer.oculus.com/documentation/unity/move-overview/ Although in my opinion the arms still look a bit janky. You can see a demo of a sample they included with the SDK here: https://youtu.be/IQt4wTdGK64?t=419

The Go can be unlocked: https://developer.oculus.com/blog/unlocking-oculus-go/

There is "passthrough", where cameras on the Oculus are relayed to the screens within, and it seems to be capable of mixing that with the virtual world too: https://developer.oculus.com/blog/mixed-reality-with-passthrough/ Also, there's a lot of innovation happening in display tech for AR too which will be even better, since you'll just see the real world more or less directly.

>They broke the emerging VR software ecosystem into the open side which many companies supported and a Facebook only proprietary one What? There used to be a bunch of different VR platforms, and only recently has the industry settled on a single open standard, OpenXR[1], and Facebook was (or at least claims to be[2]) one of the major contributors to that open standard. There are a lot of things you can criticize Meta for doing with Oculus, but opposing open standards isn't one of them. [1] https://www.khronos.org/openxr/ [2] https://developer.oculus.com/blog/openxr-for-oculus/

> Accessing or using the unlockable software (“Software”) is subject to the Oculus Terms of Service or, if you use your Facebook account to access Oculus Products, the Supplemental Oculus Terms of Service and Facebook Terms of Service (the “Applicable TOS”). For clarity, Oculus Products (as described in the Applicable TOS) include the Software. We provide the Software to you for your personal and noncommercial use only on your personal Oculus Products. Installing the Software voids all warranties, express or implied, applicable to Oculus Products. In no event shall Facebook, its affiliates or any of their respective directors, officers, employees or agents have responsibility or liability arising out of or relating to making the Software available to you. https://developer.oculus.com/licenses/go-unlock-tos Free as in beer

Like much in this announcement, the details are still to be revealed; but this is from the Developer blog : Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device. [1] https://developer.oculus.com/blog/mixed-reality-with-passthrough/

I was giving an anecdote, yo. Calm down. Synths in the digital environment of a spatial audio engine, which I've done work in both programmatically and as a musician/sound engineer, has problems with spatializing the unnatural waveforms that synths produce. Humans have trouble pinpointing where those sounds are coming from. It's easy to work around, and no one is saying "don't use synths in spatial audio", just that there are considerations to be made to make them work well in that environment. Just like you don't want to pre-apply reverb because that's going to be generated by the spatial engine. Here's a link with a bit about sine waves, but I've found the same to be true (to a lesser extend) with saw, triangle, and square waves too. https://developer.oculus.com/learn/audio-intro-sounddesign

> The real problem isn't technology or even adoption, it's that people who work at giant companies suck at making games. Many companies besides Facebook have developed games for the Quest. I guess you could say Facebook sucks at working with game developers, but I’m not sure if that’s true either. Unreal and Unity have well-developed integrations with Oculus [0]. I think where Facebook really falls down is tying customers’ FB account to their Oculus Store purchases (and ability to use the Quest at all). They’re applying a social media platform’s aggressive algorithms to suspend or ban accounts which violate their expansive TOS, which is wrong in consumer product and digital marketplace space. For that reason, mixed with personal experience at the hands of Facebook's algorithms, I have no interest in developing for Oculus. [0] https://developer.oculus.com/get-started/

> https://developer.oculus.com/blog/the-next-chapter-of-oculus-developme… Still lots of content censorship requirements for this new way to distribute. Why does content have to submit to a blanket rejection of dealing with any sexuality when it's not even going through their store? https://developer.oculus.com/policy/content-guidelines/

That misses the largest and fastest-growing market for VR: self-contained systems like Oculus Quest, which has had no real competition for about a year now. They ask that you pitch them with marketing materials and get permission before you start working on anything, so they can "compassionately" tell you not to even start: * https://developer.oculus.com/blog/submitting-your-app-to-the-oculus-qu… * https://developer.oculus.com/quest-pitch-form/ > If we feel that your app is not a fit for Quest, we'd like to let you know that early in the development process, before you've made significant production investments, rather than at the end.

my guess: oculus. https://developer.oculus.com/documentation/unity/?locale=en_US

Yes! Oculus makes an SDK for this. You can use it in Unity 3D, Unreal, or directly in a native app. https://developer.oculus.com/documentation/audiosdk/latest/concepts/bo…

For binocular vision, you can adjust the distance the object appears at virtually. Oculus gives a guideline of 0.75-3.5m for rendering objects that will have long term gaze (UI, etc). The screen itself is at a focal distance of about 1.3m, which is what your eyeballs need to focus on individually (flexing the lens itself). https://developer.oculus.com/design/latest/concepts/bp_app_imaging/

Part One is here: https://developer.oculus.com/blog/medium-under-the-hood-part-1-develop…

There's an article about this here: https://developer.oculus.com/design/latest/concepts/bp_app_imaging/ I was reading it specifically in the context of problems with displaying text in VR. Paraphrasing: in real life, your body uses two inputs to intuit distance: your eyes' focal distance and your eyes' relative angle. Focal distance is fixed inside the headset, so your body loses one of its depth cues. I suspect that contributes to the overall sense that seeing things in VR can get tiring unusually quickly.