Can we extract the contents of a *.vpk file?

[Killarny]

I just downloaded it, and I am getting the same error that tony posted a screenshot of.. should we have the program in the eq2 directory?

[Drumstix42]

Ummm this topic is about vpK files, while this program is looking for vpL files.

Might I kindly say "wth?"

*edit* and I get an error with vpL and vpK files

[tonyis3l33t]

the VPL is like eq2ui.xml you open it to see all the other stuff.

I still get error too....If the non-user friendly version works i wouldnt mind using that for now...
________
Sherri_Moon live

[Drumstix42]

Ah ok thanx for that tid bit.

[blaz]

Sorry for the trouble everyone. I'll try to fix this as soon as possible.

I thought the problem arose from the way I was handling paths. In the first version I was mixing forward and backward slashes in the path names in some cases, so I changed it in order to use backward slashes only. It works fine on the two Windows 2000 machines I have tested it on, but I don't have any Windows XP/2003 installations, so I can't really test it there. Is everyone using Windows XP here?

I'll upload a new version later today which hopefully will give a better diagnostic of what is going wrong. It also fixes a few bugs I found today (race conditions due to the multithreaded nature of the app).

A couple of notes:

- You don't have to run the tool from the EQ2 directory. It should work fine from any directory you choose.
- I agree that a VPK extractor tool should open VPK files directly, but it doesn't at the moment It will only open the VPK files that are referenced by the VPL file you choose.
- I'm afraid the non-user-friendly version wouldn't work either, because this is not a problem with the UI part of the app.

[tonyis3l33t]

aww xp users have to wait

I use XP and am willing to test whatever you want, blaz
________
ExoticBeautty live

[tonyis3l33t]

new error inserted on top of old error in above post
________
How To Roll A Blunt

[Killarny]

I used to develop windows ui apps pretty heavily, so I know how much of a headache they are - just wanted to say thanks for taking the time to make this tool, it looks like it should turn out great!

[blaz]

Thanks to the error message tonyis3l33t posted I understand why the first version fails on so many people's machines. For some reason I fail to understand, my AssetsLib.vpl file stores relative paths to the VPK files, while most of the VPL files out there store absolute paths. As I only had access to my AssetsLib file I never thought this would happen.

I have updated the tool to handle both cases:

http://eq2.blazlabs.com/Eq2VpkTool-1.1.zip

This version should be a little more bug-free Hope it is finally usable.

[tonyis3l33t]

it opened with no errors! checkin it out now good job blaz

________
teen girl Webcam

[mixer343]

heres a perl script by thelxepeia that extracts VPK files
copy this code and rename to decompress.pl
put the perl script and the *.vpk in the main perl folder (usually C:\Perl\)
from the command line example: "perl decompress.pl antonica.vpk unpacked"
and it will extract everything into the C:\Perl\unpacked\ folder

use Compress::Zlib; 

$in = $ARGV[0]; 
$path = $ARGV[1]; 

die "Missing input file name.\n" unless $in; 
die "Missing input file name.\n" unless $path; 

mkdir $path; 

open (VPK, "< $in"); 
binmode(VPK); 

$chunk_count = 0; 

read(VPK,$packed_len,4); 

while (length($packed_len) == 4) 
{ 
   $unpacked_len = unpack("L", $packed_len); 
   read(VPK,$data_seg,$unpacked_len); 
   read(VPK,$packed_len,4); 
   if (length($packed_len) != 4) 
   { 
      $file_block = uncompress($data_seg); 
      unless (defined($file_block)) 
      { 
         $file_block = $data_seg; 
      } 
  
      ($fnblocksize, $fncount) = unpack("LL", $file_block); 
      $file_block = substr($file_block, 8); 

      for ($i = 1; $i <= $fncount; $i++) 
      { 
         ($block_start, $block_len, $fnlen) = unpack("LLL", $file_block); 
         $file_name = substr($file_block,12,$fnlen); 
         $file_name =~ /(.*)\/(.*)/; 
         @paths = split(/\//,$1); 
         $path = $ARGV[1]; 
         foreach $dir(@paths) 
         { 
            $path = $path . '/' . $dir; 
            mkdir $path; 
         } 
         $out = "> " . $ARGV[1] . "/" . $file_name; 
         open(OUT, $out); 
         binmode(OUT); 

         seek(VPK, $block_start + 4, SEEK_SET); 
         read(VPK, $bin_data, $block_len); 
         $u_bin_data = uncompress($bin_data); 
         unless (defined($u_bin_data)) 
         { 
            $u_bin_data = $bin_data; 
         } 
         print(OUT substr($u_bin_data, 9 + length($file_name))); 
         close(OUT); 
         $file_block = substr($file_block, 12 + $fnlen); 
      } 
   } 
} 

close (VPK);

have fun

[blaz]

Thanks for the Perl script, mixer343. It certainly is much more lightweight than my tool when all you want is to extract everything from a given VPK file.

[tonyis3l33t]

yay i got all the sounds and they atually play back, hehe
________
Personal herbal vaporizer

[tonyis3l33t]

extracting with path informations is a bit messed up.
If you try to extract fo a folder called folder01, you will get a new folder next to folder01 called folder01environments or folder01sounds, etc.
________
Live sex

[blaz]

I see. That's a bug I had already fixed that managed to crawl back in. I'll fix it later today and release a new version, along with source code, if I have some free time.

Thanks for the bug report.

[blaz]

I just released a new version that fixes the directory naming problem.

I have also released the source code for the tool and for the library I developed to analyze the EQ2 file format. There's a lot of interesting stuff in there, more than what the tool is currently using. If you take a look at the library source code you'll see it has classes that:

• Help process VPL and VPK files, giving access to the packed files as if they were regular files on disk.
• Parse the EQ2 scripts (.vdl) and convert them to XML so that they can be manipulated by the .NET XML classes.
• Decode the format of every single type of asset EQ2 uses (zones, models, animations, particle systems, etc). This doesn't mean it knows how to display them, though

I'm trying to develop a library that can be used to help create viewers or format converters for things like models and animations. This might be of interest to someone who is coding a 3D engine and wants some great models to work with. I am in this situation

I'd like to start creating a viewer application that will let you see all those textures, models, animations, etc, that the tool unpacks; I think I'll learn a lot of stuff while coding it. It will take a while, though, so I urge everyone who is interested in something like this to go ahead and code something up themselves, using this library if they find it useful.

I will also try to start documenting the EQ2 file format, even though you can already learn a lot from the source code. There are things that are not that obvious looking at the source code, and an explanation is always welcome.

[tonyis3l33t]

making new stuff is cool, learning while you do it makes it that much funner
________
Laguna Bay II Condos Prathumnak

[Killarny]

blaz, how do you know all this about these file formats? are you privvy to information that nobody else is? or are the eq2 file formats commonly used in other games?

[blaz]

I've always been interested in learning more about formats, especially if they are not documented. I like the feeling of finally being able to make sense of something that at first looked incomprehensible. Much more so when they are trying to prevent you from figuring out how it works (i.e. by encrypting/obfuscating it). This means you usually have to resort to reverse engineering, either by disassembling the program or otherwise peeking under the hood to see what's going on.

In the case of EQ2's file format, the main reason why I'm interested in how it works is because I'd like to convert to/from other known formats, and be able to use them in my personal projects. But apart from that, I also have the opportunity to learn a lot of stuff, and I think that's important.

As for EQ2's file format being used in other games, I'm not aware of it. I'm certain it's a proprietary format.

[blaz]

I have uploaded a new version of the tool that can decrypt some of the zone maps.

The decryption algorithm is actually very simple. Take a look at the tool's source code if you want the details.

[Drumstix42]

awesome job!

[tonyis3l33t]

wow thats cool you found out some (or all?) of their decryption process
I'll see if i can think of some more features for your prog, keep ya busy
________
Dessireex

[thelxepeia]

The perl script above is just a rewrite of a python script created by daeken_bb over at eqemulator.net. Great work blaz. I’ve got a command line c++ version now as well. I’ve been working on a UI myself but it looks like you beat me to it :-( . I’m really interested in getting into the 3D files myself. It looks like I’m several steps behind you but I’m gonna try out what you’ve got here and see if I can catch up a bit. Feel free to PM me if you’d like some collaboration on this.

[blaz]

If you are interested in the 3D format, I suggest you take a look at the VeRenderMesh code first. That's the class that represents a raw mesh as stored in .draw files. The format is quite straightforward (if you ignore the likes of 'autoDropVertexIndices' and 'noPolygonShadowTriangleIndices', which I have no clue as to what they are), and you can probably have a simple DirectX or OpenGL app that renders these meshes in little time.

If you want to render the mesh with textures, it is more complex. You get the texture coordinates from the VeRenderMesh object, but you don't get the information on which textures to assign to each stage.

The process would be as follows. You would first parse the script with the definition for the object, for example:

clientvdl/appearances/accessories/wieldable_items/weapons/axe/battleaxe/battleaxe001_plain.vdl

And get the path to the VOC file with the "visual definition" of the object:

composed/characters/accessories/wieldable_items/weapons/axe/battleaxe/battleaxe001_plain.voc

You would then decode that file and end up with an object of the VeMeshGeometryNode class (or some class derived from it). This class contains information on each of the LODs of the object. For each LOD, you get the path to the render meshes that represent the object and the path to the "shader palette":

cooked/characters/accessories/wieldable_items/weapons/axe/battleaxe/battleaxe002.sp

This .sp file contains an object of the VeShaderPalette class, which in turn contains a list of scripts that define the shader palette:

shaders/characters/accessories/wieldable_items/weapons/axe/battleaxe/battleaxe001plain.vdl

Finally, this script defines the textures to apply to the object and has a reference to the shader code that is used to render the object:

shaders/templates/bclr_t2_bmp1_spwm.vdl

The problem with this approach is that there is no easy way to integrate all this script and shader code stuff in an existing 3D engine. You'd probably be better off exporting all the data you need to your own format and working with that.

If you want to work with animations, the classes to refer to are VeAnimMeshGeometryNode, VeSkeleton (.skeleton files) and VeAnimation (.anim files).

[mixer343]

Originally Posted by tonyis3l33t

yay i got all the sounds and they atually play back, hehe

i can open it up fine, but what are you using to play the mp3? winamp and wmp 10 open the file, but cant play anything.

are you using a specific codec?

EDIT:

Ugh ok i can hear the environment sounds and mob sounds, but can anyone play the voiceovers?