torstai 13. toukokuuta 2010

Naali 3d canvases & mediaurl handling

This text was originally sent to the realxtend-dev mailing list.

Thanks to Jonne's efforts recently, showing qt widgets as textures in the 3d scene in Naali is back working again. And the old textureprop/genericmessage mediaurl handling that I wrote using it in December is now enabled in develop and uses that new implementation for display.

I had a little fun with a custom use of this yesterday -- this video shows a html+js animated webpage as the texture of avatar's shirt: http://www.youtube.com/watch?v=Ikh9ZrJ_aaQ . The webpage used there is http://bubblemark.com/dhtml.htm which is good for fps testing, has an fps meter for the own drawing. In my setup had refreshrate 20fps for the 3d view, and naali on linux kept running >50fps, while the qwebview was running bubblemark with defaults at 40-50fps (external window). All looked and felt smooth, and was fun to resize the 2d view to scale the bubbles up in the texture :)

The EC_3DCanvas is basically identical to the EC_UICanvas implementations that was in the old qtmodule last year, but instead of the uicanvases we had then for both 2d & 3d display, this takes the qwidget directly as a param. But input, which worked fully with the old system back in October or so, is not yet implemented at all. Dealing with the focus etc. is the tricky part that Jukka has been investigating.

The actual mediaurl handling nor the 3dcanvas itself don't yet have any UI for opening the same widget for 2d display, where input works. My test uses little custom code that put to command.py. I leave it there, so in the upcoming 0.2.2 release if someone wants to play similarily can go to this line in pymodules/usr/command.py: if 0: #webview as 3dcanvas . Change 0 to 1, and press alt-. inworld to see it. You can change the urlstring in the code snippet there, and modify the submesh/material indexes too (default is [0], so assigns to the first mat index of the user's avatar). This test code is local only, not synched in any way, so only you see it. The 2d view of the widget this code opens is an external window.

A use case for mediaurls is where the webview is showing something like etherpad or google docs, which updates live in the inworld view, but which can also be opened for editing. We plan to use this in an inworld rex testing workshop with the French Ener team next week. The workaround with current code is to put a server side script to the same object that is having the mediaurl to send a LoadURL command when the object is clicked. That will give two separate webviews, where scrolling the 2d one doesn't affect how the 3d one is scrolled or anything. Just now realized that that is the desired behaviour probably anyway, so your inworld view keeps showing the same that the others are seeing too. But still some UI for opening any 3dcanvas for 2d view is probably good, in client side without needing custom server side logic .. with a
mouseclick or something.

How to scale, offset etc. arbitrarily sized web pages by default etc. is AFAIK a bit of an open question still. Do defaults work ok or should that be configurable? I haven't looked yet how the new shared media thing in slviewer works.

Besides the old rex mediaurl genericmessage handling, I also started writing now a EC_WebView as a test, using the new Attribute system in Naali which gives automatic net sync of component data. This would be much simpler and robust than the texture mediaurl handling, and also allow adding new features easily .. like synching the scroll offset when
desired or something.

~Toni

P.S. oh and the use case for showing something animated in the avatar's shirt is another request from Francois -- have videoconferencing work so that participants video shows in the av t-shirt, while the av face etc. remains for av animations etc. I had thought of having avatars with box heads where the video shows where the face is, but agreed that his idea is better :) .. should be easy now with the comms module video call videos, 'cause they are already shown using a qwidget. And Francois was thinking it could be also just showing a webcam image of the avatar's user, doesn't necessarily have to be naali which sends the video. And of course doesn't have to be the full shirt texture, I guess putting a video/webview plane as an attachment to the chest can work simply and well?

P.S.2 about video in these webviews: qwebview doesn't support flash out of the box, so youtube etc. doesn't work now. That's why I used the bubblemark html+js thing for fps testing. Hopefully qwebviews get html5 video tag support soon (and youtube etc add support for something else
than h264 so that open source software can show the videos?). Using a qvideowidget should work, but I don't know what formats+codecs are supported there - Matti K. etc. I guess know based on the video calls work.