Linux/Mac Version Switcher

[mattinm]

Linux/Mac Version Switcher

[EDIT] Demonstration on Youtube

[EDIT2]

Yay! I've gotten Admin approval from Elly that my version switcher is clean.

For anyone curious about installing, please see the project wiki.

If anyone decides to try this, please post any bugs on the sourceforge project page. Thanks!
[/EDIT2]

Mods, let me know if this is against policy. I don't believe it is, because it does what TrenShadow does, just within a different OS.

I recently installed D2, updated, etc on my linux partition, and it runs like a dream. However, to avoid having to create multiple install points or rebooting into Windows (to use TrenShadows version switcher), I began work on my own Python-based version switcher. If this has already been done, could someone point me to the code, please... don't wanna re-invent the wheel ... but I couldn't find anything.

It's still in a rough state, but it gets the job done. I need to work on making it more secure (double-checking so it doesn't overwrite .bak files), add more functionality (it can handle multiple windows in a round-about-way, but I'd like to add it to the script), some serious code cleanup (was more of a quick put together), and added functionality, especially inline switches such as -w, -ns, etc.

Anyway, my question is how many people would be interested in looking at the code and/or testing this. Obviously I would recommend backing up your D2 folder before undertaking this. Python is not my primary coding language (I have < 1 year experience in it). I'll make sure to have file-overwrite protection before I let anyone touch it though.

With a little code-correction and helpful testing, this could even be used on Mac (as well as Windows, but TrenShadow works fine there). So, if somebody wants to give me a hand with that by giving it a go and telling me what's wrong, that would be lovely.

 

[silospen1]

Re: Linux/Mac Version Switcher

I can have a look at it if you want, email it to [email protected].

I looooooove Python

 

[Oscuro]

Re: Linux/Mac Version Switcher

I'll chime in and mention that I've got a toy version switcher of my own; it uses unionfs-fuse and a bash script to assemble any version you want, and it keeps save files separate for each version. I can provide more info if anyone is interested.

 

[mattinm]

Re: Linux/Mac Version Switcher

@silo: Just mailed it off. All the files/directories get backed up, are copied (with sys links), everything loads up fine, and everything shutdown fine. But I'm having some troubles with the game itself, but I'm also having troubles with a non-updated vanilla install.

Edit: Figured out the problems, I hope.

Judging by the types of errors (BITCOUNT, etc), I'm think it's because I'm using 64-bit wine and a 64-bit system. I'm going to give it a shot on my netbook, which is 32-bit to see if the problem continues. It looks like it's working, however, and I'd be glad if you could give it a try and/or look at the code.

The code could still use some cleanup, and it's not very pythonic (again, not my first language), but it should work alright, and it *shouldn't* be hard to port to mac if anyone wants to help with that (screenshots, file lists, etc). My biggest concern with the port is making sure the right files are copied and the game is started the correct way.

@Oscuro: If I could see the code for that, that'd be lovely. It's always nice to see how other devs tackle problems and any ideas would be nice. If you'd like to see my code, just give me a shout. My email is marshall.mattingly(@)gmail.com

It works almost identically to TrenShadow. You create a folder with the required files (the .dll's and the Patch_D2.mpq) and run versionswitcher.py. It expects the folder its in to be the mod folder and "../" to be the game folder by default, but you can set these with --src=path_to_mod and --dir=path_to_game, respectively. If you use --dir, however, make sure you terminate the folder with '/'... I just fixed that, but it's not present in the .tar.gz I sent.

It also allows for -w, -t (or --txt), and -n (or --ns) to do, well, you can figure that one out . Well, off to the netbook for the final bout of testing for the night.

 

[silospen1]

Re: Linux/Mac Version Switcher

I'll chime in and mention that I've got a toy version switcher of my own; it uses unionfs-fuse and a bash script to assemble any version you want, and it keeps save files separate for each version. I can provide more info if anyone is interested.

Ooooh, I'd be interested in the details of that, unionfs is a great idea. I'd like to see how you implemented it.

 

[mattinm]

Re: Linux/Mac Version Switcher

Ooooh, I'd be interested in the details of that, unionfs is a great idea. I'd like to see how you implemented it.

I agree. UnionFS is probably the most elegant system you could use with something like this, for Linux at least. Much more elegant than backing up files, creating links, deleting said links, and restoring files.

 

[Ragnarod]

Re: Linux/Mac Version Switcher

Well, I've got a mac and I'd love to have a version switcher available (especially since all I'm playing right now is 1.06), but I'm not sure it would allow you to play older versions anyways.

D2 is older than the OS X macs currently use, and Blizzard released a special installer to allow D2 to run in OS X back then (I think the game version at the time was 1.09, but I didn't have a mac so I'm not certain).

So... in short, you can't play a version older than 1.09 (or perhaps 1.10) even with a fresh install in OS X right now, which probably means the switcher won't be able to make them run either, right? That said, I know nothing about programming, but if you still want to give it a try and need info about d2 files in the mac install and/or testing I'll be happy to help.

 

[sorcererbob]

Re: Linux/Mac Version Switcher

Wouldn't the best solution for this problem be a universal one? ie. one that doesn't involve UnionFS, Bash or anything else which isn't totally cross platform? At least then there is only one code base to maintain.

I'd love to see a cross-platform Python solution. Also, I've got a bit of Python coding experience under my belt (both in an educational and professional context) and I'd love to take a gander at your code - [email protected]

 

[mattinm]

Re: Linux/Mac Version Switcher

I didn't get a chance to test it in 32-bit linux/wine last night, but I'll do that here in the next few hours.

@Ragnarod: Indeed that would be a problem; still, if it can get back to 1.09 that might be useful for people wanting to do classic rushes within OSX. It's kind of difficult without having a Mac myself, but perhaps we can work together to figure it out.

Anyway, if you could send me a link containing all the files your D2 folder, that could be a useful place to start. I can send along a python program if you'd like that could just dump them to a text file.

@bob: I agree that a cross-platform solution would be useful (1 source to maintain for all versions). I probably won't be as elegant as it could be on individual systems, but it should get the job done.

I'm working on adding some more things to it, such as OS detection (Windows/Linux/Mac and version), OS specific file operations, and OS specific game calls. Obviously, the Mac part of that will be somewhat lacking to begin with.

I'll send along the code probably toward the end of the day, my time.

Edit: Looks like I found the problem in the code. I had a work-around in place where Patch_D2.mpq wasn't required, but gave a warning when it wasn't found (for playing 1.07), and it made it so Patch_D2.mpq wasn't being copied at all. Now I've been able to start characters and run around in 1.07/1.09b/1.12a.

Edit2: Just tested it out in Windows, and it works in there too . Now to clean up the code, use more try/except's instead of attempting the double-checking, and testing some more. I'm going to try and get it up on a website for people to look at... any help/ideas with the coding would be nice.

 

[jamesixgun]

Re: Linux/Mac Version Switcher

There is (afik) no way to run pre-1.09 in OSX.

Well, that's not quite true. PowerPC Macs that can run the Classic environment will run any version of D2 (with some difficulties and rather poor performance due to the implementation of "Classic OS" in the Classic environment).

Intel macs *might* be able to run OS9 through a program called SoyLatte (an OS9 virtualization thingy), but I've been unable to compile SoyLatte to test this. I'm really fairly ignorant when it comes to stuff like this. Meh.

Of course, if you want to run early versions of D2 on a mac, you could purchase a MDD PowerMac or CRT iMac and play it in OS8 or OS9, but you'd have to find proper install disks. I looked into this when I was working on the Mac Guide, and decided there were better ways I could spend ~$400.

Or you could buy a cheap PC, but that sort of defeats the purpose of being a Mac user.

Anyways, Thanks for your work on this, mattinm!

 

[mattinm]

Re: Linux/Mac Version Switcher

Thanks for that info jaygun. Sill, if it's possible to get 1.09 and/or betas available on OS X couple be nice for those users. It would at least allow them to do quicker HF rushes and have access to gfg beta runewords (again, if this patch is available on OS X).

I'm not saying people would do that, but it's a nice option to have, at least, without having to take up excess HD space. I'm gonna start doing some digging on the subject and try to acquaint myself with the differences. I'll come back if I have any specific questions.

I'm just finishing up some touches, but I'm just about ready to release this on my freeweb along with documentation/instructions.

[highlight]Notable Features[/highlight]

• Cross-Platform compatible! - Linux and Windows currently
• Open Source - I'm still debating on which license I'm going to use, which I've got to decide before I post it up
• "Start in" - Starts in the mod folder itself, protecting your primary data folder
• Reg Saves Windows Only - Changes the 'Save Path' registry value so the save folder does not have to be copied (this is what TrenShadow does)
• Sym Saves *nix Only - Backs up the main save folder (save.bak) and creates a symlink to the mod folder to minimize file movement; deletes symlink and renames save.bak on exit.
• Command-line Arguments - You can pass --src, --dir, --txt, --ns, -w, and --help currently.

[highlight]Workarounds[/highlight]

• multiswitcher.py allows the running of multiples of the same version (doy!). It is currently pretty stupid (you must run it in the same mod folder as versionswitcher.py; it only checks for Game.exe.bak to run) -- I'm looking to incorporate this directly into versionswitcher.py.

I'll edit this post when I put it out there and get confirmation from the mods that I can indeed link to it.

 

[jamesixgun]

Re: Linux/Mac Version Switcher

iirc, the OSX native installer patches to 1.10(whateverthefinalversionwas). I tried a couple of workarounds, but I don't think I ever got 1.09.

I might be wrong though.


(I went and looked at the MacGuide, and it is indeed 1.10.)

 

[mattinm]

Re: Linux/Mac Version Switcher

@jaygun: Blast! Hmmm, I'm sure that someone would have come up with a way to do it by now if it could be done. I'll try some google-foo, but I'm not overly optimistic. Poor Mac users.

 

[Kugel]

Re: Linux/Mac Version Switcher

Hi,
I actually had the same idea to write one in Python. You beat me to it. I will be glad to test it or contribute if needed.
I haven't used TrenShadow's but how do you handle the big files? Just swaping them, or doing binary patching?
Also let's say I have 1.12 installed, do you include 1.09 version files with that? Or do I have to gather them manually?

 

[mattinm]

Re: Linux/Mac Version Switcher

@Kugel: There are only certain files that get changed in updates (around 7 MBs, I think). I have versions copies of all the folders for 1.06b (classic) and 1.07 through 1.11b (including the betas), and I'll put them on my site in .zip and .tar.gz formats as soon as I get permission to link to it.

The script simply backs up the files it will change in the main folder, then moves the ones in the current folder over.

The data folder (which will get propagated if you use the --txt switch) fills to a few MB, and a data folder in the main D2 folder could potentially be much bigger. Thankfully, the script changes the working directory (if it has to) to the mod folder itself and calls "/path/to/Diablo/Game.exe" from there. This means D2 will use the data folder in the mod directory and not even have to worry about the one in the main folder.

The saves are handled by registry editing on Windows, meaning nothing at all has to move. In *nix, it copies the main save folder to save.bak (renaming should be fast) and creates a symbolic link to the save directory in the mod folder. This, again, means that no files actually have to be changed.

 

[Oscuro]

Re: Linux/Mac Version Switcher

Here is the code in its entirety. This does not reflect the installation process, which I did by manually mounting/unmounting the directories as needed.

The premise is that we keep a 'pristine' copy of each version which we never directly run the game from, and have one or more 'working' directories for each version. The versions are layered to create a full installation, such as:

1.09b working
1.09b pristine
1.07 pristine (LoD)
1.03 pristine (Base classic version on my cd)

Pristine directories are mounted read-only just for good measure. It's easy to create a fresh instance of any version by creating another directory and adding an entry to the script. The whole lot takes 2.9g of disk space since no data is duplicated.

#!/bin/bash

cd $HOME/.wine/drive_c/Program\ Files

# Unmount Diablo II if it is mounted
fusermount -u Diablo\ II

if [ -e "Diablo II/Game.exe" ]
then
	echo "Problem unmounting D2 directory"
	exit
fi

case "$1" in
	"")	echo "Usage: d2 VERSION"
		echo "e.g. d2 1.09b"
		exit
		;;

	"1.03")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.03=RW":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.04b")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.04b=RW":"$(pwd)/d2installs/pristine/1.04b=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.04c")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.04c=RW":"$(pwd)/d2installs/pristine/1.04c=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.05")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.05=RW":"$(pwd)/d2installs/pristine/1.05=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.05b")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.05b=RW":"$(pwd)/d2installs/pristine/1.05b=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.06")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.06=RW":"$(pwd)/d2installs/pristine/1.06=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.06b")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.06b=RW":"$(pwd)/d2installs/pristine/1.06b=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.07")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.07=RW":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.08")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.08=RW":"$(pwd)/d2installs/pristine/1.08=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.09")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.09=RW":"$(pwd)/d2installs/pristine/1.09=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.09b")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.09b=RW":"$(pwd)/d2installs/pristine/1.09b=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.09d")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.09d=RW":"$(pwd)/d2installs/pristine/1.09d=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.10beta")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.10beta=RW":"$(pwd)/d2installs/pristine/1.10beta=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.10sbeta")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.10sbeta=RW":"$(pwd)/d2installs/pristine/1.10sbeta=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.11")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.11=RW":"$(pwd)/d2installs/pristine/1.11=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.11b")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.11b=RW":"$(pwd)/d2installs/pristine/1.11b=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"1.12a")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/1.12a=RW":"$(pwd)/d2installs/pristine/1.12a=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

	"mxl")
		unionfs-fuse -o cow "$(pwd)/d2installs/working/mxl=RW":"$(pwd)/d2installs/pristine/1.12a=RO":"$(pwd)/d2installs/pristine/1.07=RO":"$(pwd)/d2installs/pristine/1.03=RO" Diablo\ II
		;;

esac	

if [ -e "Diablo II/Game.exe" ]
then
	cd Diablo\ II
	wine Diablo\ II.exe
else
	echo "Error with unionfs-fuse"
	exit
fi

edit: @mattinm You can manipulate the registry with Wine as well if you want for the save files by using regedit to apply a Windows-style .reg file to Wine's registry.

 

[mattinm]

Re: Linux/Mac Version Switcher

@oscuru: Thanks for that code. That's pretty much how I figured you'd do it. It really is a very elegant solution to this type of problem.

As to the reg editing in Wine, I'll look at it tonight. I'll have to check, but if the registry write locations in the windows format (C: \Program...) instead of the physical location (~/.wine/drive_c/...), it could cause some issues.

I'd have to figure out the conversion of physical location (moddir) to windows/wine format. I'll give it a look either way and see if I can figure something out. I'd prefer to use native solutions where possible, especially considering the Mac variation should be the exact same as the Linux variation (except for files). But, then again, there could be a more elegant way on Mac too.

Well, this got a little long-winded . Once I'm finished up with some stuff in Windows, I'll reboot to linux and give some stuff a try.

 

[mattinm]

Re: Linux/Mac Version Switcher

I've got a nice new feature implemented: Multiple window support. I don't really want to talk about how I do it anymore, but rather what it does. If I get mod approval to link to the site, anyone will be able to look at the code, but I'll just talk about theory and implementation for now.

When the script (for lack of a better term) is ran, it simply copies the files over and runs just like TrenShadow. This, however, has a command prompt that pops up (if it wasn't ran from there to begin with) and shows some diagnostics as well as allowing new features while the game is actually running.

Currently I've only implemented the "new game" command. You simply press n (or N) in the command box and it opens a new game. It keeps track of all the games and when all of them are closed, it automatically does it's cleanup.

If you accidentally click the script twice to open new games, it will tell you that it the files are already backed up and remind you how to open a new game.

The code works, but it's still not very Pythonic; I want to get it working in a more C fashion (as is my primary language), then work on making it look fancier.

If anyone has any ideas that they would like to see implemented, I'll try to work on them. I'm not really looking for any volunteers to actually do coding, or to even discuss coding, until I get permission to link to the script. Thanks for your help!

 

[Thyiad1]

Re: Linux/Mac Version Switcher

Mod making and mod support has never been done in the SPF. It's always done at The Phrozen Keep. The link is still in the Stickied FAQ; I think I copied it wholesale from AE's. As mattinm is fairly new, he may not have known that when this thread was posted.

However, because this is a similar situation to GoMule/ATMA (update cross platform), we have left this thread for people to provide ideas to mattinm. If you want to have a discussion on how to code, then that's really more appropriate to PM or MSN.

The major issue sirpoopsalot and I have is that neither of us use nor have access to a Mac. So when it comes to verifying the program we are utterly reliant on others. From that point of view it's going to take us a little while to be able to give a go ahead on posting links to this.

 

[mattinm]

Re: Linux/Mac Version Switcher

I have progressed to the point where I'm very comfortable with the code [highlight]for Windows[/highlight]. I haven't done extensive testing within Linux, and I haven't even attempted Mac specific code yet (with some posts, I'm almost certain it would be a fruitless venture).

That said, I'm looking for anyone willing to beta test the Windows functionality. The program does some (imho) pretty neat things, such as inherent (and safe) multi-instance support, opening multiple instances with a single shortcut (by passing a flag), and warning the user (and stopping the program) if there is a potential problem (i.e. files failed to revert when closing), and telling the user to run with the -r flag to repair the installation.

If anyone is willing to beta test for me, while I'm checking the Linux side, please send me a PM, and I'd be very appreciative. I've done about 2-3 hours of testing with the current code-base in Windows, but, of course, you can never be 100% certain how it will react on other systems.