Naali 3d canvases & mediaurl handling

This text was originally sent to the realxtend-dev mailing list.

Thanks to Jonne's efforts recently, showing qt widgets as textures in the 3d scene in Naali is back working again. And the old textureprop/genericmessage mediaurl handling that I wrote using it in December is now enabled in develop and uses that new implementation for display.

I had a little fun with a custom use of this yesterday -- this video shows a html+js animated webpage as the texture of avatar's shirt: http://www.youtube.com/watch?v=Ikh9ZrJ_aaQ . The webpage used there is http://bubblemark.com/dhtml.htm which is good for fps testing, has an fps meter for the own drawing. In my setup had refreshrate 20fps for the 3d view, and naali on linux kept running >50fps, while the qwebview was running bubblemark with defaults at 40-50fps (external window). All looked and felt smooth, and was fun to resize the 2d view to scale the bubbles up in the texture :)

The EC_3DCanvas is basically identical to the EC_UICanvas implementations that was in the old qtmodule last year, but instead of the uicanvases we had then for both 2d & 3d display, this takes the qwidget directly as a param. But input, which worked fully with the old system back in October or so, is not yet implemented at all. Dealing with the focus etc. is the tricky part that Jukka has been investigating.

The actual mediaurl handling nor the 3dcanvas itself don't yet have any UI for opening the same widget for 2d display, where input works. My test uses little custom code that put to command.py. I leave it there, so in the upcoming 0.2.2 release if someone wants to play similarily can go to this line in pymodules/usr/command.py: if 0: #webview as 3dcanvas . Change 0 to 1, and press alt-. inworld to see it. You can change the urlstring in the code snippet there, and modify the submesh/material indexes too (default is [0], so assigns to the first mat index of the user's avatar). This test code is local only, not synched in any way, so only you see it. The 2d view of the widget this code opens is an external window.

A use case for mediaurls is where the webview is showing something like etherpad or google docs, which updates live in the inworld view, but which can also be opened for editing. We plan to use this in an inworld rex testing workshop with the French Ener team next week. The workaround with current code is to put a server side script to the same object that is having the mediaurl to send a LoadURL command when the object is clicked. That will give two separate webviews, where scrolling the 2d one doesn't affect how the 3d one is scrolled or anything. Just now realized that that is the desired behaviour probably anyway, so your inworld view keeps showing the same that the others are seeing too. But still some UI for opening any 3dcanvas for 2d view is probably good, in client side without needing custom server side logic .. with a
mouseclick or something.

How to scale, offset etc. arbitrarily sized web pages by default etc. is AFAIK a bit of an open question still. Do defaults work ok or should that be configurable? I haven't looked yet how the new shared media thing in slviewer works.

Besides the old rex mediaurl genericmessage handling, I also started writing now a EC_WebView as a test, using the new Attribute system in Naali which gives automatic net sync of component data. This would be much simpler and robust than the texture mediaurl handling, and also allow adding new features easily .. like synching the scroll offset when
desired or something.

~Toni

P.S. oh and the use case for showing something animated in the avatar's shirt is another request from Francois -- have videoconferencing work so that participants video shows in the av t-shirt, while the av face etc. remains for av animations etc. I had thought of having avatars with box heads where the video shows where the face is, but agreed that his idea is better :) .. should be easy now with the comms module video call videos, 'cause they are already shown using a qwidget. And Francois was thinking it could be also just showing a webcam image of the avatar's user, doesn't necessarily have to be naali which sends the video. And of course doesn't have to be the full shirt texture, I guess putting a video/webview plane as an attachment to the chest can work simply and well?

P.S.2 about video in these webviews: qwebview doesn't support flash out of the box, so youtube etc. doesn't work now. That's why I used the bubblemark html+js thing for fps testing. Hopefully qwebviews get html5 video tag support soon (and youtube etc add support for something else
than h264 so that open source software can show the videos?). Using a qvideowidget should work, but I don't know what formats+codecs are supported there - Matti K. etc. I guess know based on the video calls work.

Naali on the Nokia N900

If your not a big reader skip to the bottom and at least check out the video!

...

I bought a Nokia N900 phone recently. It's a great device and has a lot of potential with the linux based Maemo5 OS and the processing power. First I made some simple small application in Qt to test how it's done and learned a lot. The thing was that these application didn't do or show anything that could be useful for anyone.

Hmm... why not build Naali for that thing? Is it too heavy to run, what about the 3D rendering? What about building Ogre and all the other dependencies? Only way to find out is to give it a try. I had a 4 day easter holiday (extended weekend) coming up so I started the home project then. I made slow but steady progress during these days. Building ARM versions of Ogre and all the deps were the biggest challenge for sure.

Naali itself didn't need that much work, just disabling some modules and I got myself the first build. As an end result I had Naali that started from the terminal and opened a QMainWindow that had ogre rendering to it. The UI was not showing, the window was just black. Oh... I also had one pissed girlfriend for not spending the holiday with her. She watched/listened me swear and curse on the computer for days, thanks Susanna for putting up with me ;)

On the next workday I showed this "thing" to my co-workers. Didn't get that good of a welcome as... well it didn't do anyting. So next few days I worked on actually showing something and doing a login. I made a simple timer into rexlogic to blast a hardcoded login away when Naali was done loading. And what do you know, it went in and showed me a bringht yellow Audi world. As I later discovered it was the EnvironmentModule doing this, it also crashed the viewer every time terrain was fully loaded. So I disabled the module, after that everything started to load and most importantly look good with the GLES rendering. Of course the sky was black and terrain had no texture due to disabling the module, but our target world is indoors so that didn't matter at the moment.

Next I showed this thing at work and then I got some peoples interest peaking. Then it became my job to make this thing "working" and in demo shape in 1-2 weeks. I took the challenge, did my best and here is the result.

The video demos everything I've managed to do so far specifically for the Maemo platform. This includes removing the normal Naali UI and remaking a simple UI for the interactions that we were interested in, without obstructing the world view too much. Some small adjustments in the window creation. I also tweaked the input system to work with a touch screen and made some special camera controls. Probably I missed something, small changes were required here and there.

Check out the video at

http://www.youtube.com/watch?v=IP7zty5Kg7g

Best Regards,

Jonne Nauha

realXtend developer

Using Facebook chat with Naali

Since Facebook opened its chat to Jabber/XMPP clients, it has been available to be used with Naali. Now you already can have your Facebook friends list in Naali and chat with people with it. Voice chat, however, doesn't work, because the Facebook chat doesn't support it. You can view your jabber id and server settings from the Facebook guide.

New UI in development

The new Naali platform is still some way behind the original realXtend viewer when it comes to feature completeness. It seems that our original predictions were correct, however, and the new technology enables us to create new features and fix issues at a high rate. One of the aspects we will be looking into a lot more in the near future is the user interface.

Until now UI design has been driven by the need to have some kind of a way to access the functionalities mostly for testing purposes. After the release of the 0.1 version (Don't forget to pick yours here) we will be concentrating a bit more on the UI side. Here's a brief explanation on the new approach by Jonne Nauha (typos are all mine):

Ether concept

We have had some actual UI design meetings now and decided on some high level consepts. One of them is "ether space" versus "inworld space". Th e aim is to have "ether" as the view where you land in when you launch Naali. It will be the place to make logins, create new avatars, create new worlds and possibly access tools/settings for the viewer. We would like to get rid of the "write your information to these text fields" approach. Of course this is somewhat necessary when creating avatars /worlds, but in basic use the workflow should be "I'll pick this avatar, I'll pick this world, ok lets go" -type of thing.

We tried to think of a simple and at the same time nice looking and inviting thing, so we started prototyping a "flip book" type of thing that resembles iTunes cover flow, but for virtual worlds. Me and Mikko Alaluusua have worked on this for a few days now and have got something together already. We did this out of Naali codebase with QtCreator and used pure Qt, namely the Qt animation framework is used heavily. The cards get loaded into a scene that is shown in a full screen graphics view, so it will be easy to port into Naali.

The fps in my capture software was too low to show the animations fluently and somehow it altered the colors too, but you should be able to get the idea. The navigation is WASD or arrow keys, and we'll add mouse selection functionality to the "cards" too. The background image and other graphics are only for prototyping, not what will eventually be included in Naali. For now the avatars have the basic image of the naali (arctic fox) and worlds all share the globe image. The idea with the images is that your avatar's screenshot will be inserted there and it is updated when you change your appearance or logout (not really decided yet) and for the world images a screenshot is taken from the position you last left the world in question.

Inworld widget interaction

The inworld widgets will probably stay with the slim frame or maybe some styling will be done there with proper graphics. The main thing inworld would be to get rid of the age old toolbar at the top of the screen. We are now close to 10 widgets and they will only increase in numbers when we go forward. We were thinking of giving each corner inworld a certain type like widgets on the top left, notifications / status things on the top right, messaging on the lower left etc. Of course the positions or the final look are not firmly decided yet. But at least I would like to remake the "widget toolbar" into something icon based and add some snappy animations like some kind of a layered menu that has sub groups for widgets like interacting, world building etc.

Also some kind of workspaces would be nice to make. So you could make different UI views and swap between them fast to get to your tools. A little bit like operating systems have multiple desktop views that have their own set of applications and you can move apps between them.

UI for interacting with objects and people

We also need something similar to pie menu we all know and love/hate :) There hasn't been any decisions about this yet, but some kind of a hovering "info card" for things you want to interact is the goal. This infocard could follow the avatar / object that you have interest in or it could be pinned to a position. The idea for these cards would be that there would be a viewport (camera) to the item, for example the avatars face that you clicked or movable view of a object you are interacting with. Also this card would have quick selections for sending messages to a user etc. This will also be the most challenging thing if we do it purely on Qt that needs an ogre viewport rendered to it. Right now all info that is showed of people is a static 2D overlay that has the avatars name. And if this is the best we can do with ogre then we probably go with Qt on these infocards.

Schedule

The aim is to get most of this done by the march release. It may be that "ether space" will have 70% of the defined features, inworld "info cards" 50% etc. as its hard to know what obstacles there will be ahead. Especially when our Ogre+Qt is a special thing without paint events that there always can be something unsuspected outcomes of this, good example is that we had fully working video calls in Naali IM before we moved them inside the scene and now gstreamer doesent render the video to the window handle.

Naali 0.1(first public rc) released!

This is it. There are still a number of bugs to fix (some of the code heroes sitting right next to me are currently hard at work fixing and tweaking) and an incredible number of new features to do, but as the version number 0.1 suggests more than a final product this really is a glimpse of things to come. Nevertheless and although I might be considered partial, I still think it's already quite interesting and suggest you try it out for yourselves. Let's take a quick look.



Swimming with the fishes is as pretty as ever. There's of course a lot of room for optimization, but the graphical capabilities have traditionally been one of the focus points in our development and that won't change with the new Naali viewer. You can connect to existing realXtend and OpenSim worlds (sorry, walking between regions is still in the works) or host your own with the realXtend server (new version to be released soon).

Here's a shot of the user interface. We're currently working on enhancements to the visuals and a new layout for the UI, this is an example of some of the functionality. Note that I'm logged in with my Jabber account, which makes it possible for me to communicate with the outside world as well. Currently it's also possible to make video calls (and only video calls, yes, we're working on it) and we have quite a few ideas about future development for the communications module.

Ok pay attention, everyone, because now comes the really important part! We started the work on the new viewer and server software because we had a lot of ideas on how to improve realXtend, but the software architecture we had at the time didn't offer the kind of room for expansion we were looking for. Naali does. And not only room for us to work with, but for everyone interested in it as well. realXtend is an open source project where everyone can participate. The code is available under the liberal Apache 2 license, grab yours today right after you test the binary we prepared for you.

We would love to have your input on what you think about the new viewer as well as the bug reports that will inevitably be generated now that more and more people start testing the software. From now on we intend to make new binary releases more often, which means you will get to see the new features relatively quickly even if you don't want to build the code yourself. The Naali viewer is designed to be cross-platform and we have run tests on Linux, Mac support is on the way as well. I hope there are some enthusiastic Mac and Linux users who would like to join the community and together make sure Naali works equally well on all the platforms.

Get the software here:
http://code.google.com/p/realxtend-naali/

Here you can find discussion on realXtend software:
http://groups.google.com/group/realxtend
http://groups.google.com/group/realxtend-dev

If you run into bugs, please let us know here (please check for known issues first):
http://code.google.com/p/realxtend-naali/issues/list

We will be explaining the new features in more detail on this blog soon, you might also want to browse the existing documentation here:
http://wiki.realxtend.org/index.php/Main_Page

Tutorial to import scene from Blender to realXtend

Requirements:
Blender
Ogre mesh exporter plugin (http://www.ogre3d.org/wiki/index.php/Blender_Exporter)
Ogre scene exporter plugin (http://www.ogre3d.org/wiki/index.php/Blender_dotScene_Exporter)
ModreX trunk or branch 0.1.2-rc



First you need to export all meshes from the scene. Select all meshes in scene by pressing 'a'. Then select OGRE mesh export tool from File->Export->OGRE Meshes. Ensure that you have selected all meshes from the "Selected" box. Select your favorite options and ensure that you have selected at least options "Fix Up Axis to Y" and OgreXMLConverter. OgreXMLConverter option may require that you specify the path to your OgreXMLConverter in the properties section.



In some cases not all textures are copied to the destination directory. To export textures from .blender file select File->External Data->Unpack into files. This will extract all textures from scene to location of your .blender file.



After exporting meshes you need to also export the scene file. This includes information on where the objects are placed in the scene. Select scene export tool from File->Export->OGRE Scene. In this dialog, ensure that all meshes are selected and that "Fix Up Axis to Y" is NOT selected.



Now after exporting scene and mesh files, we are ready to import them. Copy scene-, material-, mesh and texture files to OpenSim bin folder. Ensure that the name of the scene and material files are same expect for the file extension of course. Boot up your simulator if it isn't already running. Apply offset if necessary with command "ogrescene offset x,y,z", where x, y and z are the offset values of x, y and z axis. X and Y values of the objects in Scene file need to be between 0-256m and Z needs to be higher than 0. Otherwise the objects won't show up. By applying the offset command you can affect these values. Note that usually waterlevel in OpenSim is at 20 meters. Import scene with command "ogrescene import X", where X is the name of the scene without the .scene file extension.



After inputting the import command it may take a while to import all the objects. Sometimes there might be errors when importing the scene or when exporting to OGRE format and objects might not view exactly the same as in Blender.


Notes by Antti: The above example .blend file was retrieved from this site: http://gryllus.net/Blender/StudentGallery.html

Also note that the export tools for blender are quite specific about the format of the textures. You need to create UV maps for the textures and have the TexFace selection on in the Blender material menu.

ModreX region in OSGrid

This is a feature that many people have asked for: the ability to use grid mode with ModreX. We have now made this simple test server for testing interoperability of ModreX regions in an OpenSim grid.



Getting the region in ModreX mode is a pretty straightforward procedure. First configure your region according to OSGrid instructions, then modify ModreX configurations. First thing to do is to enable the default Event Queue and disable RexEventQueue. Without this, once entered to ModreX region, one can't teleport or walk out of it. Second thing to set is the client stack plugin. Set clientstack_plugin configuration to "ModularRex.dll" instead of the default LL. This makes UDP listener spawn RexClientViews instead of LLClientViews. This is important when using realXtend features like meshes, particle scripts and so on.

Final thing to notice from ModreX options is the ClientView option. By default this option is in mode compatible. The compatible mode is ment for interoperability purposes, it supports both LL and realXtend clients. However there is a drawback, compatible mode doesn't support smaller avatar update messages. This means that clients spend twice as much bytes on avatar update messages in compatible mode than in Naali or 0.4 mode. Both Naali and 0.4 mode do support these smaller avatar update messages, but SL client crashes on these messages, so these modes shouldn't be used in grids where SL clients are also used. Differences with Naali and 0.4 modes aren't that great, but as you can expect 0.4 supports the older realXtend clients and their features and is also compatible with Naali clients. Naali mode contains a bit more experimental packets that aren't handled properly in realXtend 0.4 series clients.

Now that the region is configured it can be added to the grid. Although you can see from the picture that the region is working in the grid it is still lacking some features. User needs to press Shift+R (on the 0.42 realXtend viewer) to switch to OGRE rendering and to view the scene properly. Currently grid mode doesn't support realXtend authentication or realXtend avatars. Also at time of the testing there were some problems uploading textures and the physics engine crashed few times. Mediaurls are disabled from the client since it didn't receive rex_mode flag in login response.