Creating 16 bit maps for TA3D

[Balthazar]

Since TA Annihilator is onep-source map editor for TA is it possinble to upgrade it to use 16 bps textures for map creation?
Matman will create several hi-color maps for us in this case What do you think, Zuzuf?

http://tauniverse.com/forum/showthread.php?t=40937

[zuzuf]

I think 32bits maps would be better because we could have 16777216 colors instead of 65536 and an alpha channel that could be used to store additional information like transparency for a space map or a density of something like fog or some other atmospheric effect.

[Balthazar]

I was saying abot 16 bits because of size of maps. 32 bit`s maps will be twice bigger than 16 bits ones, and they are even 8 bit can be quite big (20-30 MB)! But support of 32 bit maps a good thing too ;P

[zuzuf]

There is a big difference between 8bits color indexed images and RGB(A) images : with the later you can use lossy compression (like JPEG, JPEG2000) which are pretty effective to reduce data size

[Balthazar]

Then yeah, why not

[xpoy]

/:^}~!

UP

If it wasn't a really big project, I can try in this maybe.
Cause still had 1 week free time
But what's point is I had no idea in how to do this. I can write code down if some idea ...

[zuzuf]

That's a good idea and it should not be too difficult as it's not a core part of the engine (I mean you don't need to know all the code to add support for some extensions to the TNT map format).

Basically all you need on the TA3D side is in the tnt.cpp and tnt.h files:
http://svn.ta3d.org/ta3d/trunk/src/ta3d/src/

I don't know much about the map editor's code so you know as much as me.

Anyway the first step is designing the extension (thinking of a good way of implementing it) so you'll have to read the TNT specs:
ftp://downloads.ta3d.org/misc/OTA%20fil ... fmt-v2.txt

Feel free to ask me any questions concerning the code

[xpoy]

I want make a new file format for this, but OTA didnt' support tho /:^|
Will make same interface as tnt.h
and maybe change a TAE or re-create one.

But every step will post in here.

file format soon.

[xpoy]

So we get a format frame after consider some days, I think it should include those:
magic sign, current map version, mapinfo(width, height, sea level at moment), offset of palette(0 for 32 bits color), offset of anim table, offset of anim structs(in map),

offset of standard mimamap, offset of height table, separator magic(color in every pos array after this)

so it can be a format like this:

Code: Select all

pcode, c style
char= 8 bits, and int= 32 bits, long long= 64 bits.


//-------------------- The offset is always start from file start. 
//                     When someone isn't, will mark it out.
struct tagNewMapFormat
{
	unsigned int ("TA3D");
	unsigned char ub_MapFormatVersion;// 0 for alpha, 0x1-0xf for beta, 
                                  //0x10 and more for release version, 
                                  //backward compatibility
	struct mapinfo minfo_CurrentMap;//mapinfo
	unsigned int ui_OffsetPalette;  //0 for 32 bits, 
                                //otherwise it was offset of palette
	unsigned int ui_OffsetAnimTable;// offset of Anim object struct array
	unsigned int ui_OffsetAnimArray;// offset of Anim's pos struct array
	unsigned int ui_OffsetMimaMap;  // think it can be a picture format, 
                                //like jpeg/ bmp/ or what
	unsigned int ui_OffsetHeightTable; // same as its name
	unsigned short int ("UN");
        unsigned byte ub_Data[1];// this array's size define by mapinfo's width* height and palette. It just save color of map pos in linear
} NewMapFormat ;

//-----------
/*    here is the struct of detail    */
//

typedef struct mapinfo
{
  unsigned int ui_Width;
  unsigned int ui_Height;
  unsigned int ui_Level;
  unsigned int ui_reserve;
} MapInfo;

typedef struct Position
{
  unsigned int ui_X;
  unsigned int ui_Y;
  unsigned int ui_reserve;
} Position;

//------------
//Palette array start as ("PALE"), end as ("ELAP")
//The Palette Element is locate by sub, so a sub member is useful. but more for alignment
typedef struct tagPaletteElement
{
    unsigned byte ub_Red;
    unsigned byte ub_Blue;
    unsigned byte ub_Green;
    unsigned byte ub_CurrentSub;
} PaletteElement;

//---------------
//Anim array start as ("ANIM"), end as ("MINA")
typedef struct tagAnim
{
    unsigned int ui_CurrentObjectSize;// the size of current struct, 
                                      //include all of this struct. 
                                      //But there are some spec value.
/*
  0xffff ffff== zero truncated style string, for ARTub_data is anim object name;
  0xffff fffe~ 0xffff fff0== reserve
*/
    unsigned byte ARYub_data[1];//the array size define by ui_CurrentObjectSize
}

//-----------------
//Anim pos array struct start as ("AMIA"), end as "AIMA"
typedef struct tagAnimPos
{
   struct Position pos_CurrentAnim;
   unsigned int ui_AnimSub;// this sub used in locate the anim in anim objects array
} AnimPos;

//------------
//Mimamap live as a struct

typedef struct tagMimaMapInfo
{
  unsigned int ("MIMP");//magic 
  unsigned int ui_CurrentStructSize;
  unsigned byte ub_Data[1];// the size define by ui_CurrentStructSize
} MimaMapInfo;

//----------
//Height is a array of HeightPos Struct
// height live as this:
// a array of height pos struct, when last pos's height is same as current pos, avoid current pos' height pos but check next pos's height, so loop to the height be change, and 

just save change height pos. When the X loop to 0, always record the one height pos;
typedef struct tagHeightPos
{
  struct Position pos_CurrentPos;
  unsigned short int height;//do we need that much height?
}

[zuzuf]

Magic words may be useful for debugging but they use some extra space so they should not be required in order to have a valid map file (except the first magic string ID which identifies the format of course).

Code: Select all

        unsigned byte ub_Data[1];// this array's size define by mapinfo's width* height and palette. It just save color of map pos in linear

This can be quite big, remember width and height are in units of 16pixels. For instance:

Code: Select all

with = 768 (in blocs of 16pixels)
height = 768 (in blocs of 16pixels)
bpp = 32bits (R,G,B,alpha)
size = (768 * 16)² * 4 = 576 MB!

I took those numbers from a real map. As you can see this is really too much, OTA compresses its maps by indexing repeated tiles (if you have a map with a small number of tiles you can divide the map size by 1024 !) and since maps are still designed more or less in the same way it could still be reused (this may not be useful for rendered maps ...). You could also compress tiles using a lossy compression method like JPEG, JPEG2000, ... using only subsampling without further analysis would already reduce map size by a factor of 2. Special care would be required for things like "alpha" channel (or atmospheric channel ?) if we intent to use it to describe some special effects like volumetric fog, heat effect (would be cool on a desert or lava map ), light emission, transparency (having a space background for a space map would be awesome ), ...

[xpoy]

well, I'm poor in game design for sure.
And is this be support?
A spec value for a pos, which describe the type of the pos, like up and down able to pass (some cave need this tho)/ water/ lava/ etc.

I didn't want make it as pixel, but as position, which save as terrain type, it can be some customize or native terrain type builder, it work like read the type of terrain, then use a base picture and change a bit, then draw down to the spec position. When position's size is no idea..
8 magic number just used 4*8 32 bytes , it's useful in make sure things going true but a bit size reduce

for the fog, well.....some addtion effect type in position? And this illustrate my damn negligence, must had a mothed to had some position to together, I mean some big anim or some big fog in terrain...

[zuzuf]

You can use flags, TNT format has some unused fields that could be used .

I am a bit lost, where do you store the terrain type description/tiles/whatever may be referred to by ub_Data ?

[xpoy]

well this can be a new format but load as filling into a TNT format maps. But perfer a same interface as current tnt format but not transform to TNT
format update in tommorw, will write code when it be suitable.

the terrain type will store in a array of position struct, as well as addtion effect in position, and hign 4 bits of addtion effect flag will used in position's group, what same X coor or Y coor and a bits in hign 4 bits be same, it be a group of positions, maybe used in future or what....
when anim's info just store in anim's array.

[zuzuf]

I like the idea of grouping similar positions it can greatly reduce map size

[xpoy]

/:^J I didn't like work as this, this talk more than do. But must work as this for make sure what will do is good.

So we used a mass type group flag for filling in same things, which reduce size in purpuse.

And a rough circuit of read this map:
1. read file
2. check the valid and check the step be need for load, when there can be no anim, or no palette(more colors than 16 bits, I make it as 24 bits color /:^).
3.load terrain data and generate a corresponding coordinates terrain struct array from group flags and maybe some other.
4.generate terrain texture (or what for draw terrain? hadn't read detail of tnt infact ) by terrain position builder, with palette or not. 5.then generate the addtion effect in terrain, which hadn't got a idea in what to generate for TA3D -------*
6.at now, the terrain be done. so Anim array be load and generate a anim from load- anim name's file or inside anim object data -------*
7.when get the anim array, fill it into terrain by anim pos struct array ------- *
8.finally get the map in memory, which can be used by TA3D.

maybe some of this will change, but atm, will try to get a loader for TA3D(and for test... ) at first, then a writer for editor

[zuzuf]

I think you should develop the loader and writer at the same time : you'll need both if you want to test something

[xpoy]

I brb some days. in train tho. so had much time in consider about map loader and writer.

[MattyWS]

So whats up with this anyways? I want to make some maps! Another thing to consider really is the ability to use models to create bridges and the like. Not sure how much of that is to do with xpoy though. _

[zuzuf]

I don't know how bridges work in OTA, maybe it works in TA3D too. Has someone tested ?

[MattyWS]

Bridges in TA are impossible unless it's part of the height map. Because OTA didn't allow units to use models as a 'ground' to walk on. I'm hoping theres a possible way with your engine, to allow me to make models of bridges in the maps. Even platforms that units can stand on.

[zuzuf]

In current state it probably won't work the way I see it but it's possible to edit the walkable area maps used for pathfinding, then crossing the bridge could be done by scripting but editing the height map may be too complicated since it is used for rendering too. Maybe having an hard coded property of model collision for bridge units would make it possible without scripting and without editing too much the map data.