1

Topic: Is there any roadmap?

Is there any roadmap available for Wired?
HFS+ Resource fork support is planned? It seems, that the protocol itself doesn't support forks.

Re: Is there any roadmap?

No particular road map per se, I take suggestions and requests as they come. Presently in progress is 1.1, which includes the tracker. Resource fork support is not currently planned.

3

Re: Is there any roadmap?

I don't get it. The one good reason I can think of to write yet another alternative to Hotline for Macs is to finally have a server that handles Mac filesystems. As far as I can tell, none of the others do, as if the ability to handle HFS+ was trivial, and not the single biggest deal-breaker for people thinking about switching to a non-Hotline server.

I have a huge archive of Mac software. I'd love to move to this server/client system, I like the look and feel, but how can anyone use a server that won't serve resource forks and metadata? What's the point of all the other enhancements if half your files get corrupted?

Or is everyone else just using this software to server linux and windows systems?

Re: Is there any roadmap?

daguerre wrote:

Or is everyone else just using this software to server linux and windows systems?

My guess is that they are encoding their mac files in .sit format.

Re: Is there any roadmap?

What do the resource forks do? Do they really matter?

Re: Is there any roadmap?

airwave wrote:

What do the resource forks do? Do they really matter?

On a Macintosh, there are two forks available for a file to use. Not all files contain a resource fork. Many (like a text document, or a file from a PC using Windows or Linux) have only a data fork. A data fork holds data (like the contents of a text document), while a resource fork holds categorized data for the application itself or application that created the document.

The resource fork is constructed in a similar fashion to a non-relational SQL database. It contains resources (or tables as in the SQL simile) of a specific type that hold a single or multiples of that type of data. Below is an ascii representation:

Resource Fork of an application
        |
        +------> Resource type 'vers'
        |                     |
        |                     +------> 'vers' data ID 0
        |                     |
        |                     +------> 'vers' data ID 1
        |                     |
        |                     +------> 'vers' data ID 2
        |
        +------> Resource type 'icl8'
                              |
                              +------> 'icl8' data ID 128
                              |
                              +------> 'icl8' data ID 65534

As shown above, an application can access its own resource fork and obtain the data in the following object oriented fashion (demonstrated in pseudo-code):

getResourceData ( myResourceFork().resource('vers').id(0) );

Whereas in the data fork, one must read in a linear fashion and use byte offset identifiers (or termination characters) to seperate data.

On a Macintosh, the resource fork and data fork are stored on the filesystem as being a single file. You must use the Operating Systems (OS) Application Programming Interface (API) to access the resource fork stored in the file system (HFS/HFS+ on a Macintosh). If you try to store a file that contains a resource fork on any other file system (such as NTFS for WinNT/WinXP/Win2K, FAT32 on Win95/Win98, ext2/ext3/reiserfs/xfs on Linux, or HPFS) the resource fork will be lost because almost all file systems only store data forks. On Macintosh, all files have a data fork, so they are at least compatible with other file systems. However, imagine if an application (which stores it's machine executable code in the data fork) loses its resource fork; it may run, but it will but will not be able to read any resources it has stored in its resource fork). Another boon is that Macintosh has not store executable code in the data fork for over 10 years. Only 68K applications (meant to run on old-world macintoshes with a 68000+ processor from motorola created and popularized before the first G3 chip), Nowadays, the executable code is stored in the resource fork (PowerPC code).

Apple has realized this as a potential problem and has now moved (in Mac OS X) to the .app architecture created for NeXT (owned by Steve Jobs and the precursor to Mac OS X post Rhapsody). The .app architecture incorporates old resource fork data inside a folder and in data fork files. Given the same ascii representation above, in the .app architecture it would look the same but using files and folders. So the 'icl8' id 0 data would be inside the data fork of a file stored within the .app folder of the application (you can see this by right clicking on a cocoa application and choosing "Show package contents". This also allows Mac OS X to operate on file systems such as UFS (DVD content is stored in this format as well). Wishingly this could mean that Macintosh could some day operate under the same platforms that Windows does.

For now, the only way to safely transfer files with resource forks is to encapsulate both forks into a single data fork (encoding them with Stuffit for example to make a .sit file). Where you can make a single archive like a .sit, which stores all the file data in the resource fork, transfer it over the internet, then require the end-user to decode the archive and restore the original file with both forks. Demonstrating the other benefit of the .app architecture that an application (since it has no resource forks) can be compressed in say, a zip or rar archive (which does not save resource data).

Hope this length explanation has been helpful. Cheers, devin.

Re: Is there any roadmap?

I think it's more important for a server system like Wired to support .app/.pkg than resource forks, per se.

Re: Is there any roadmap?

Thanks, devinteske! Much appreciated. :D

Did that take a long time to write?

Re: Is there any roadmap?

I absolutely agree with Arlo. Symlinks, permissions, and folder hierarchy must be preserved across uploads and downloads so that users aren't baffled when they are unable to upload or download applications and installer packages that haven't been compressed.

Carracho does that correctly, I might point out.

Re: Is there any roadmap?

When designing Wired, I did not take resource forks into accounts, because I consider them outdated starting with OSX. Do people still use them? Is resource fork support a show-stopper for you? Let me know.

Regarding .app/.pkg - these bundles are handled like folders in the lower levels of the file system and show up like folders in Wired, which means you can download/upload them like folders.

Re: Is there any roadmap?

morris wrote:

Do people still use [Resource Forks]? Is resource fork support a show-stopper for you? Let me know.

In fact, I might add, it wasn't until Mac OS X 10.3 that the Finder itself was a native cocoa application. So prior to that, the Finder was a Carbon application with a Resource Fork. Any carbon application will have a Resource Fork, morris. People sharing carbon applications will definitely appreciate the support. At this time, they are stuck with having to encode the file before-hand.

Re: Is there any roadmap?

morris wrote:

Regarding .app/.pkg - these bundles are handled like folders in the lower levels of the file system and show up like folders in Wired, which means you can download/upload them like folders.

The problem is, symlinks break, rendering them essentially useless. Carracho actually doesn't have this working 100% correctly; if Wired did, it would be a major draw for people to switch.

Re: Is there any roadmap?

devinteske wrote:

In fact, I might add, it wasn't until Mac OS X 10.3 that the Finder itself was a native cocoa application. So prior to that, the Finder was a Carbon application with a Resource Fork. Any carbon application will have a Resource Fork, morris. People sharing carbon applications will definitely appreciate the support. At this time, they are stuck with having to encode the file before-hand.

Actually:
1. The Finder is still a Carbon application.
2. Carbon applications can have their resources stored in data-fork resource files. The Finder does this, and I exepect all other Apple Carbon apps do as well.

Re: Is there any roadmap?

Uncommon wrote:

1. The Finder is still a Carbon application.

Hmmm, thought I had read that somewhere as a new feature of Panther.

Uncommon wrote:

2. Carbon applications can have their resources stored in data-fork resource files. The Finder does this, and I exepect all other Apple Carbon apps do as well.

Well it's fine to expect Apple Carbon apps to do it that way, but how many people are going to be sharing Apple Carbon apps? Non-Apple Carbon apps however I would hardly expect to do, because why go through the trouble of writing a Carbon application which can run on Mac Classic and Mac OS X and then butcher it (spinning the resource fork off into the data fork) so that it will not work on Mac Classic? The developer might as well have created it in Cocoa.

Re: Is there any roadmap?

I'm pretty sure OS 9 can handle data-fork resources as well.

Not that I don't think it's important for Wired to handle resource forks. I just wanted to correct a couple of minor misconceptions.

Re: Is there any roadmap?

All you need to do is stuff the file, and wired won't butcher it.

That said, I'd prefer that uploading .apps would work - i.e. privs, symlinks, etc. should all be preserved.

Re: Is there any roadmap?

noah wrote:

I'd prefer that uploading .apps would work - i.e. privs, symlinks, etc. should all be preserved.

There's only one privilege that you really only need to be concerned about and that is the executable bit (bit 1). Other than that, you should really follow the umask of the system (or offer a umask for your application for newly created files). You can throw the file ownership out the window from a client because the user on the client will most definitely not exist on the server (unless by some freak of nature, or if the file is owned by root, which in that case, you definitely don't want to maintain file ownership because that could be a security risk). You don't really have to worry about Apple-based privileges (OS 9 style in the Get Info window of a file/folder) since when they are omitted, they inherit from the UNIX permissions.

As far as symlinks breaking, I personally have not encountered many symlinks within application packages but I can understand that it is not inconcievable. The question is... should uploading of symlinks even be allowed? First off, a symbolic link is a file system feature (although at it's lowest form, it is actually a text file with the target path stored inside it, with a filesystem attribute set to signify that it is a symbolic link). What the client should do is resolve symbolic links perhaps and upload the target. Another concern is that it is not guaranteed that the server file system would support the symlink (or alias, or shortcut or hard link, etc.) from the client. For example, (duh) if a client on Mac OS X uploads an alias (security risk right there) to a server running on Linux, it won't work (besides the fact that Wired doesn't handle resource forks, the Linux box would have to have access to and store the data on an HFS[+] partition, since not only would the resource fork have to be intact, filesystem wise, but the kIsAlias flag would have to be set on the filesystem).

Re: Is there any roadmap?

Eeek... do NOT resolve the symlink and download the target, at least not without that being a non-default option. Otherwise, you can think you're downloading a few megs and end up downloading gigabytes instead.

I agree, ownership should NOT be preserved.

Symlinks should be preserved, NOT followed. That way, you preserve the form of what exists on the server. If it's got relative symlinks whose scope is retained within the app bundle, then you're fine. If they go outside the app bundle, then HOPEFULLY they point to something that will exist already on your system. Otherwise it breaks. I think that's a reasonable compromise between inadvertant downloading of lots of random stuff, and breaking applications.

Re: Is there any roadmap?

I was referring to UPLOADING, not downloading. When a client uploads a .app with symlinks, perhaps the symlinks should be resolved first.

How do you go from "a few megs" to "a few gigs"??? I doubt that there are that many symlinks (to such large files) in a .app.

About symlinks breaking, there's absolutely nothing you can do about that. If the path doesn't exist on the client, then too bad. I personally don't see anything wrong with resolving links. Imagine if an admin wants to link to a folder on a second hard drive, when a client wants to download the folder, you saying that you would prefer it to transfer the symlink (how asanine)?

Re: Is there any roadmap?

No, symlinks should normally be resolved for uploads and downloads.
However, within an .app bundle, I think it should be treated specially.

Here's a contrived example: A (poorly designed) BBS program, call it UnWired, is an .app bundle, but requires that the .app bundle contain a folder that is the server filesystem. Fortunately, UnWired allows you to use a symlink or alias to this folder, so it can live elsewhere. If you upload a copy of the app, it should NOT resolve the symlink/alias to your whole filebase.

I agree, I've seen very few symlinks within app bundles, but I have seen a few. If the symlink is to something that should normally exist on the system anyways, then resolving the symlink wouldn't be desired behavior.

I don't know... I guess it seems like it should be user-definable behavior, just like with cp (-r vs. -R). Of course, this just gets into one of the major problems with OS X - the existence of both aliases and symlinks, with different behaviors. But I guess I think that uploading/downloading stuff to Wired should behave like a finder copy - preserve the links, don't resolve them. Certainly preserve file mode (not just executable bit, but read and write as well... but probably NOT setuid/setgid... I don't even want to THINK about special files of course) and certainly not file mode.

Re: Is there any roadmap?

noah wrote:

Certainly preserve file mode (not just executable bit, but read and write as well... but probably NOT setuid/setgid... I don't even want to THINK about special files of course) and certainly not file mode.

Finder copy does not preserve all attributes. It uses the system defined umask (see `/private/etc/profile') for newly created files. If you want to preserve all attributes, you do a `cp -a'.

As for the example of `UnWired' with the link to the files within the application package (thus being a horrible implementation, a better setup would be a preference file or configuration file with the path stored or an alias resource in the bundle to the files)...

After giving it some thought, I think the proper solution would be to never resolve links for file/folder transfer. This has worked for eons in both Finder copy and Stuffit as well (by default, links are not resolved in Aladdin Dropstuff). Furthermore, as to not confuse the user, all links to files should have proxy icons (the little arrow at the bottom left of the icon) indicating that it is merely a link. Links however, should be resolved for directory listings and navigation. You should be able to navigate into a link that points to a directory. When [a client is] downloading a directory, links should not be resolved (with the exception if the initial directory itself is a link pointing to a directory, but not within that). When a client downloads a link to a file, perhaps it should be resolved (why would a client want to download a link?). Another smooth ploy would be to display an icon for broken links in the file listing so that users are aware that it is broken.

As far as application packages... the Finder wishes to show them as a single file, even though it is actually a folder. So if we stick to the rule that we do not resolve links on a recursive download of a directory, then you should be fine.

With these rules in place, you should be fine. However, you may get people trying to take advantage of that if the rules are set in stone. You'd have administrators creating directories full of links so that they can download the files one at a time (because it will allow a download of the target) but not download the entire directory (because the links will not be resolved). To solve this obvious loop-hole that can be taken advantage of and abused, you can simply prompt the user each time that they request to download a directory ("do you wish to resolve any links within the directory?"). For power-users also, you should add a keyboard modifier to imply `yes' (and make `no' the default). For example, if highlighting a directory and clicking a download button recursively downloads a directory (prompting if you want to resolve links), then holding option while clicking the download button will imply that they want to resolve links within the directory, thus supressing the dialog).

22

Re: Is there any roadmap?

In my opinion, support for HFS+ resource fork is needed. A lot of carbon applcitaions, old fonts (true type, bitmap, etc.) still exists in the Macintosh world. Image files still use resource fork to store thumnails.

Taracque