Some of this has been said, some of it hasn't. I'm also not 100% on everything here, but please discuss. This is poorly understood and needs documenting in order to really progress on the issue. Too many people I've heard saying "bah windows is too much of a pain" - but go onto rubyforge, and look at the OCI download numbers. Whatever you're feelings, that's far from irrelevant. The fact that only a handful of rubyists are actually capable of building ruby + all of ext/* is quite sickening, really (at least to me). On 2 Jan 2008, at 13:02, Gary Wright wrote: > No point in keeping this discussion in the Zed thread... Absolutely. I have been thinking about raising this topic for a long time, but I'm also working on fixing some parts of it, so I was holding off until I have time to complete that work. I guess though, as it's here, now is as good a time as any. I've approached a few people for assistance on the ideas in the past, but we're all busy trying to survive ourselves to really get things going (yet). Even with a lack of time, if some people want to group together, I'd be more than happy to donate what I can to getting a more stable environment up on the Windows platform - and that includes code I've written against the issues. > I could be wrong, but I sense a deeper question in Thurfir's message. > > What is it about Linux (or *BSD or Mac OS X) that avoids all > the compilation problems that arise in the Windows environment? > > Can a Windows expert confirm my understanding that: I am by no means an expert, but I have been studying this recently, and have come to some understanding of the issues. I'm not much of a C/ C++ programmer, but I have done a lot of building, and sent / committed patch files to build processes of quite a number of projects. I also actively use Windows, *nix (in many flavours), and more recently OS X. > Unlink Linux (and *BSD, Mac OS X...), Windows does not come packaged > with a development environment and in fact there are multiple > mutually-incompatible development environments (Cygwin, MinGW, VC6, > VC8). For the most part you can not mix-and-match object code (dlls) > created by these different environments with the biggest problem > being that there is no common memory allocation library. Well, actually OS X doesn't come "pre-packaged" either, but the build chain there is based on Open Source stuff from the *nix world, and for ruby, most of the pre-requisites are already available as libs (readline, curses, openssl, zlib, etc). Once you install Xcode, you pretty much get a production grade compiler setup for Ruby that will Just Work. Under Windows, you can compile 1.8.6 trivially on any of the compilers mentioned. Ruby herself, in 1.8.6 is a very very clean compilation and Just Works. The ext/* stuff however, is a different story. OpenSSL compilation on Windows is actually a real pain - you need to get a copy of perl if you want to use the MS compilers (if you're using the pre-packaged build chain that is), and under MinGW you have to rebuild the directory structure by hand, as the symlinks in the tarball will not expand properly. I don't know why the OpenSSL devs ship their build packages like this, I find it terribly annoying, but it is what it is. As for readline, well it's broken on win32, and several of the builds out there are archaic. GnuWin32 is useful - but again, stupid choices make life difficult, such as the pre-compiled binaries being hardcoded (!!!) to particular locations on disk. It's just insane - and this isn't the fault of the operating system - this is third party craziness. Other libs have similar issues, I won't got through the whole ext/* stack now. zlib is easier. Getting just rubygems + ruby up and running is quite easy, if you use one of the many pre-compiled openssl builds out there and build + link against that. As for the MSVCRT issues, I have read many many mixed reports. The summary is roughly like this, AFAIK: - You *can* mix msvcrt versions safely *if* you don't pass structs / pointers across the boundaries for which the api has changed. In reality, for some applications, this is unworkable - however there are several production ready gems out there that are working just fine on the OCI (MSVCRT) which are compiled with VS 2005 (MSVCR7/8 (I can't remember the exact version -> VS pairings, sorry). - Due to the above, and several apps having issues with it - as well as the bloat it causes, no one likes mixing up MSVCR versions. That's fair enough. - I have been told by several MS developers (whether or not this is true, someone with more knowledge of the linking strategies and A[PB]Is will need to confirm - that it's possible to make these things safe using particularly compiled .lib files. I've also heard stories of shims being put in place, but they may come with a performance overhead, and are complex low level things in this domain (I suspect also quite specific to particular problems). - MinGW and MSVCRT (VS 6) are compatible. - Performance on MinGW is up about 30% over VS6, for 1.8.6. The gap is closing in 1.9, and isn't so bad for later MS compilers. - I haven't tried Intels compiler, and someone probably should, although it's buyware. - MinGW has cross-compiler capabilities, and that might be worth noting here. I know there are a couple of gems out there that are built like this. If possible, it might make sense to make something available for building pre-compiled gems like this. The problem is, C toolchains and dependancies are such a PITA very often. - In other build environments, I have built software that doesn't link against the standard runtimes locally at all. I need to become more familiar with the build tools and ABI to say more than that though. - There are bugs in the 1.8.6 ext/* makefiles which means that subshells spawned seem to use different relative / absolute paths in different ways. In order to compile everything off of a single 'make' under mingw, I found I had to add -I [absolute_path] AND -I [relative_path]. I think the same errors have thrown quite a few people who later just gave up. IIRC it's openssl vs. readline that don't compile properly - and those are a harder two anyway, so people get angry and give up very rapidly. - The version of bison in MinGW doesn't build parse.c properly anymore :-( There is absolutely *nothing* stopping the ruby community getting into MinGW and helping everyone out by providing better build chains for them based on ruby apps and capabilities. Seriously, rubys core interpreter (1.8.6) builds really easily on top of the standard lib under mingw and other OSes, and isn't packed up as share/buy ware like the most commonly use perl / python builds on win32 - that's important to some people (whatever you might think of that as a merit yourself). More than this though, a dose of pragmatic powerful scripting capability could go a *long* way in helping MinGW move forward, so if anyone is interested, take a serious look at a project like this. Even just bootstrapping the default required mingwPORT.sh files would go a long way into making the MinGW build process simpler. What actually needs "fixing" is we need someone to release a build chain for the build chains (for ruby herself). I'm working on something, but I have not had time to finish it, as I'm in a busy startup at present, so it's only good for internal use right now. I want to try and get into working with the rubygems folks once I'm more familiar with the chains and issues. Also interesting to join forces with might be the multiruby and related projects (there's a build-farm type project somewhere too but I can't remember the details, Ryan?) > I assume this is why some Windows software comes distributed with > its own collection of 'standard' dlls compiled in the same > development environment as the underlying application. Well, XP comes with two really, a native api, and the MSVCRT. As you install software, at some point you'll get an app with newer MSVCR versions being installed from the re-distributable. There are disgusting rumors about GPL violations and other complex licensing issues with linking against their newer libs - but this is just so totally anti-pragmatic. I found the Wireshark toolchain documentation very good - and something we should aspire too as a base minimum. > The end result is that the least-common-denominator for all these > environments is source code, which still requires the 'end-user' to > install a development environment of some sort and to manage the > compile/link process for Ruby and for any and all 3rd party > libraries/gems/extensions that they need. This is actually true of all the operating systems. It's very common to have it there already on *nix, and many developers install Xcode very early on in their configuration of a new OSX box, as this is required for fink / macports / general open source software compilation. This would be very similar on win32 if only mingw's build chain wasn't so nasty at this point in time. > I also gather that each of those environments has very different > build utilities (the Windows equivalent of make, config, autoconf, > and so on). For an extension writer, the problem is that they can't > really know what the build process is going to look like on an > arbitrary Windows system making it very difficult to Well, sure, but most of that actually works for a C app, at least, there are good example builds around that people can "borrow". Going into C++ is a different matter (there are more compiler and api differences there - i'm also less familiar though), but lets start with the most common first.