Title Creating a Compressed, Self-Extracting Installation File
Author Andre v.d. Merwe dart@iafrica.com
Published March 1998 - Pinnacle Delphi Developer
Copyright Reprinted with permission from Pinnacle Publishing, Inc. http://www.pinpub.com
Downloads
  Back to the Zen Home Page



Intro:

In this article I will show you how you can create your own self extractor. This article deals specifically with a self extractor for use in an instillation package. Once you have seen how to write a self-extractor you can write your own self extractor to do whatever it is that you need.

This article explains the various components of a self extractor, these are

 

Some terminology

Self extracting exe

Finished exe, contains both the extractor and data to be extracted. This is the file that is distributed.

Extractor / self extractor

Code that is used to extract the files appended to its exe.

Maker / SE Maker

Application that creates the self extracting exe

 

The first thing that you need to be aware of is that you can append data to a windows exe (executable) without affecting the exe in anyway. You can also open an running exe for read only access. These two facts are what make writing a self extractor possible.

What happens is that the SE (self extract) maker, takes the files to be included, compresses them adds some additional information. The resulting data is appended to the extractor. When the extractor is run it scans for appended data, if this data is found the files are extracted and setup.exe is executed.

As you will shortly see a self extractor makes quite a bit of use of offsets and lengths in the data. A good hex editor is indispensable as you will be able to manually check that the format of the self extracting exe is valid. There a a number of great hex editors on the net, look at the usual software sites.

 

 

Self Extracting Exe - Overview:

The self extracting exe is made up of

  1. Extractor
  2. Appended data block
  3. Message for user
  4. Magic number used to indicate a valid se file

 

When run it..

  1. Checks for magic number
  2. Extracts files to a temporary directory.
  3. Runs setup.exe and waits for it to terminate
  4. Deletes all files created during extracting.

 

 

SE Maker - Overview:

The SE maker must

  1. Compress the files
  2. Create a data block
  3. Append data block to the extractor
  4. Append magic number to exe

 

 

Data Block Overview:

The compression method used is of little importance. What is important is that the self extractor can extract all files from the data block appended to it. For this reason each files size and name must be saved along with the file in the data block.

There are many different formats for a data block of this nature. Here is the format that I have chosen.

 

Figure 1 Format of the self extracting exe.

 

 

Final SE File (self extracting exe)

SE Code

Extractor

Sections

 

Length of sections

 

Message

Used as a prompt

Length of message

 

Magic number

Indicates that data has been appended

 

Sections

Number of Sections

Extractor

Section 1

 

.

.

.

 

Section n

Used as a prompt

 

Section (one for each file)

Length of section

Does not include name or length of name

Length of file name

 

Name

Name of file to extract the data to

Compressed file data

 

 

 

Lengths and Offsets:

As you can see there are a number of 'blocks' in Figure 1. Each block has an associated length, for example the file name is preceded by the name's length. You could also have a table of offsets into the file, this kind of table would act as a rudimentary FAT (file allocation table). However I feel that this just adds unnecessary complexity for something so simple. You decide..

You may notice that some lengths are saved before the associated block (eg length of file name) and some are saved after (eg length of message). Each length value is located so as to make parsing the data easier. For instance if the message length was saved before the message it would be necessary to scan through the message and have some way of telling where the message starts and then read the length value. This concept will become clear as you see how the data block parsing works.

 

 

Extracted Files:

As previously mentioned this article explains a self extractor for software instillation. So when files are extracted they should be stored in a temporary location, and deleted as soon as they are no longer needed. The extractor included in this months subscriber download, extracts the files to a temporary directory in the windows temporary directory. Once the setup is completed the directory and all files and directories that it contains are deleted. This ensures that there are no files / directories left unnecessarily on the users machine.

 

 

The SE Maker:

Figure2 is a picture of a SE maker. This application takes a number of files, a prompt and the extractor and creates the final self extracting exe.

Figure 2 Sceen shot of a SE Maker

These tasks can be divided into 4 groups

  1. Write extractor
  2. Write sections, one section per file
  3. Write user message (prompt)
  4. Write the magic number

 

SE Maker - Write Extractor

The extractor must be copied and written to a new file. This new file is the beginning of what is to become the final self extracting exe. Delphi's TFileStream greatly simplifies using files, performing complex file operations is made easy. MemoryStreams do the same for dynamically allocated memory.

Start the self extracting exe by creating a FileStream for it.

OutFile := TFileStream.Create( 'OutFile.exe',

fmOpenReadWrite or fmCreate

);

This creates a new file and opens it in read/write mode. OutFile remains open throughout the 4 stages of creating the self extracting exe.

 

Step 1 requires that the extractor be copied to the beginning of the self extracting file. See figure 1 to confirm this. Copying from one FileStream to another is very easy,

Extractor := TFileStream.Create( '..\Extractor\Extractor.exe',

fmOpenRead

);

OutFile.CopyFrom( Extractor, Extractor.size );

Extractor.Free;

 

 

SE Maker - Writing Sections

Refer to figure 2, the extractor has just been written so now the number of sections must be written to the output file, ie at no 2 of Figure 2. Writing an integer value to a stream is accomplished with a bit of type casting

OutFile.Write( pointer(@iNumSetions)^, sizeof( integer ) );

This gets the address of the integer to write, treats it as an untyped pointer and passes the dereferanced value to TFileStream.Write. This method is very handy and easy enough to understand, you'll see it quite a bit throughout this article.

The SE maker is now at number 3 of Figure 1, and must therefore write the length of this section to OutFile. The problem is that the length of the section is not yet known, as the file has not yet been compressed. So compress the file before continuing.

Once again, the compression method used is of no concern, you need not even use compression if you don't want to. The example defines a simple compression interface (SECompress.pas) with two member functions - Compress and DeCompress. You can change the compression method by changing the SECompress unit. No other changes to the SE Maker, or extractor will be necessary.

To make things easier the SECompress unit uses TMemoryStreams for compression and decompression. One for the input data and one for the output data.

 

 

Create the memory streams and load the data

mstrmData := TMemoryStream.Create;

mstrmInFile := TMemoryStream.Create;

mstrmInFile.LoadFromFile( sCurrentFileName );

Where sCurrent file is a string containing the name and path of the file currently being processed.

 

 

Next compress the loaded data

Compress := TSECompress.Create;

Compress.Compress( mstrmInFile, mstrmData );

Compress.Free;

mstrmInFile.Free;

 

TSECompress is defined in the SECompress.pas unit. At this point the compressed data is in mstrmData and the self extractor can continue writing to the outfile.

Write the length of the data

iTmp := mstrmData.Size;

OutFile.Write( pointer(@iTmp)^, sizeof( integer ) );

Notice that it is necessary to use a temporary integer variable.

 

Figure 1 number 4 and 5 require the SE Maker to write the length of the file name and file name of the current file. The file name is needed so that when the data is extracted the extractor knows what the original file name was.

iTmp := Length( sCurrentFileName );

OutFile.Write( pointer(@iTmp)^, sizeof( integer ) );

OutFile.Write( PChar(sCurrentFileName)^, iTmp );

 

 

That done its on to number 6 of figure 1, write the compressed data.

OutFile.CopyFrom( mstrmData, mstrmData.Size );

mstrmData.Free;

 

Having done this the current section is complete, if there is more than one file then numbers 3 to 6 are executed one for each. See figure 3 which shows the order in which the numbers of figure 1 are visited.

 

Figure 3 Self Extracting exe creating order

 

Lastly, the final step of writing the sections, write the total length of all the sections

OutFile.Write( pointer(@iSectionsLength)^, sizeof( integer ) );

 

 

SE Maker - Writing The Message

iMsgLen:= Length( sMessage );

OutFile.Write( pointer(@iMsgLen)^, sizeof( integer ) );

OutFile.Write( PChar(sMessage)^, iMsgLen );

 

 

SE Maker - Write the Magic Number

iNumber := MAGIC_NUMBER;

OutFile.Write( pointer(@iNumber)^, sizeof( integer ) );

 

MAGIC_NUMBER is a constant. You can basically choose any integer value, the examples use 1030431278. Just select a really large integer, you could always use the same magic number as I do. The need for a magic number will be explained shortly.

You now know how to write a SE Maker. However a SE Maker with no extractor, to extract the data is not much use. So onto the extractor.

 

The Extractor

The self extractor is an application that has the data to be extracted appended to it. It is important that this application be small. There is no point compressing files and then having a huge extractor. For this reason it is impractical to use the Delphi UI (user interface) units, (Forms, Dialogs etc…). Just including Forms.pas will mean a minimum file size of somewhere around 100k. For most extractors this is unacceptable.

To create the smallest possible extractor, short of resorting to assembly (yes it is possible!), you would only include Windows.pas and other WinAPI wrappers. Natrally the compression unit you are using must also only use these units.

If you can live with a slightly larger executable, you can then use classes.pas and SysUtils.pas. These units define the Delphi stream classes and some useful memory managment procedures. Using these classes will add about 30kb to the extractor, you must decide if the 30kb is worth it or not. For this article and the examples I will use these two units as they make everything a lot easier to understand.

Delphi 3 users also have the option of using run-time packages, if you are using run-time packages then this size issue does not concern you. The problem is that users who run a self extracting exe, created with packages enabled, must already have the Delphi 3 packages installed on their system. Many users may find downloading approximately 1mb of support files unacceptable. Once again choose the method that best suits you and your users needs.

 

The extractor must

  1. Check for the magic number
  2. Display message
  3. Extract files
  4. Run setup application
  5. Cleanup

 

 

Extractor - Magic Number:

The extractor must have some way of checking if data has been appended to it. If you happen to run the extractor when no data has been appended to it, who knows what the extractor would try to extract. The method I use to indicate to the extractor that data has been appended to it is to write a integer value at the very end of the appended data. The extractor scans to the end of its exe and if the last integer value is equal to the magic number it can continue with the extracting.

To check for the magic number the extractor needs read access to its own executable, this is not a problem

ThisExe := TFileStream.Create( ParamStr( 0 ),

fmOpenRead or fmShareDenyNone

);

ParamStr( 0 ) returns the name and path of the current application. Since the extractor is not using Delphi's UI classes it can't use Application.ExeName. Notice that the file is opened in read-only mode with sharing enabled. This is required, if you try to open a running application in another mode you will only succeed in getting an error.

 

Scan to where the magic number is stored (number 10 of figure 1)

ThisExe.Position := ThisExe.Size - sizeof( integer );

 

Read the magic number

ThisExe.Read( pointer(@iMagicNumber)^, sizeof( integer ) );

Then compare the value read with the magic number. If the value just read is not equal to the magic number, display an error message and exit. If the value is the magic number then extraction can continue.

 

 

Extractor -The Message

Most current self-extract and install applications display a short message and allow the user to cancel extraction. This prevents accidental installs and keeps most users happy.

Since the message can be any length, memory to store the message must by dynamically allocated. Remember that all strings passed to WinAPI functions must be NULL (nil or #0) terminated.

ThisExe.Position := ThisExe.Size - (sizeof( integer ) * 2);

ThisExe.Read( pointer(@iMessageLen)^, sizeof( integer ) );

szMessage := StrAlloc( iMessageLen + 1 );

The code above scans to where the length of the message is stored (number 9 of figure 1), reads the length and allocates memory for the message. The additional byte is for the NULL terminator.

 

Next, scan to the beginning of the message (number 8 of figure 1), read it, and NULL terminate.

ThisExe.Position := ThisExe.Size -

((sizeof( integer ) * 2) + iMessageLen);

ThisExe.Read( szMessage^, iMessageLen );

szMessage[iMessageLen ] := #0;

 

 

The WinAPI function MessageBox is used to prompt the user.

Result := (MessageBox( 0,

szMessage,

'SE',

MB_YESNO or MB_ICONQUESTION,

) = IDYES);

StrDispose( szMessage );

 

This code snippet will show a message box with a question icon and the message as the main text. The message box will have 2 buttons 'Yes' and 'No'. If 'yes' is clicked then result = true, else result = false. Having shown the message there is no further use for it, so the memory used to store it is freed.

 

 

Extractor -Extracting the Files

Figure 4 shows the order the extractor follows when it is executed.

 

Figure 4 Order of extractor

 

 

Get the length of the sections block (figure 1 number 7)

ThisExe.Position := ThisExe.Size -

((sizeof( integer ) * 3) +

iMessageLen);

ThisExe.Read( pointer(@iSectionsLen)^, sizeof( integer ) );

 

 

Then, figure 1 number 2, read the number of sections

 

ThisExe.Position := ThisExe.Size -

((sizeof( integer ) * 3) +

iMessageLen + iSectionsLen);

ThisExe.Read( pointer(@iNumberOfSections)^, sizeof( integer ) );

 

 

The extractor is now at number 3 of figure 1, and has thus just entered the loop that will extract all the files. Each iteration of this loop will…

  1. Read section length
  2. Read file name length
  3. Read file name
  4. Read file data
  5. Decompress data
  6. Write decompressed data to disk

 

 

Read section length

ThisExe.Read( pointer(@iLenSection)^, sizeof( integer ) );

 

Read the file name

ThisExe.Read( pointer(@iFileNameLen)^, sizeof( integer ) );

szFileName := StrAlloc( iFileNameLen + 1 );

ThisExe.Read( szFileName^, iFileNameLen );

szFileName[ iFileNameLen ] := #0;

 

Read the file data into a temporary memory stream

mstrmData := TMemoryStream.Create;

mstrmData.CopyFrom( ThisExe, iLenSection );

 

Decompress the data

mstrmDeCompress:= TMemoryStream.Create;

DeCompress := TSECompress.Create;

DeCompress.DeCompress( mstrmData, mstrmDeCompress );

DeCompress.Free;

mstrmData.Free;

As explained earlier, TSECompress is a class defined in SECompress.pas which acts as a simple interface to whichever compression method you choose.

 

 

Write decompressed data to disk

mstrmDeCompress.SaveToFile( sTempPath + '\' + szFileName );

StrDispose( szFileName );

mstrmDeCompress.Free;

sTempPath is a string containing the path of the temporary directory where the files are to be extracted to. Once this has been done for every file the extraction is complete.

ThisExe.Free;

 

 

Extractor -Run the Setup Application

After extraction is finished the extractor runs a file named "setup.exe". The setup.exe application does the actual instillation. It creates the necessary directories and copies the files to their final locations, in other words all the things a setup application is expected to do.

The setup.exe application included as part of the example, does nothing but show a message box. While this message box is visible you can use explorer to verify that all the included files have been extracted correctly.

It is obviously important that the extractor wait for setup.exe to finish, before it starts deleting the extracted files . If the extractor did not wait, setup.exe would be trying to install files that did not exist.

In the example CreateProcess and WaitForSingleObject are used to make the extractor wait for setup.exe to terminate. If you want more information on these two API calls look at the example and the Win32 help file.

 

 

Extractor - Cleanup

Removing extracted files is a very important step. If you don't delete every single temporary file, you will soon start getting hate mail from your users, if you have any users left that is… The example uses a recursive function to delete every directory and file in the temporary directory, as well as the temporary directory itself. This allows setup.exe to create files and directories in the temporary directory. When the extractor cleans up, these files and directories will also be removed.

 

 

Summary:

Well that is how you would write a self extractor. The example available from this months subscribers downloads page include a fully functional extractor and SE Maker. Feel free to use, upgrade or modify it in any way. The example was written using Delphi 3, so Delphi 2 users might have a problem opening the .DFM files. If you have this problem get Dr Bob's Delphi 3 to Delphi 2 converter from http://www.drbob42.com in the downloads section.

The source code for the extractor and SE maker is also included, and well documented, so you should have no problems understanding it. The compression unit is freely available in the SWAG archives and you can use it freely in any of your applications.




 All information on these www pages is copyright (©) 1997 Andre .v.d. Merwe And may not be copied or mirrored without my permission.

have no problems understanding it. The compression unit is freely available in the SWAG archives and you can use it freely in any of your applications.




 All information on these www pages is copyright (©) 1997 Andre .v.d. Merwe And may not be copied or mirrored without my permission.