神刀安全网

FUSE for Windows/Cygwin now available

Release v0.12 – FUSE Compatibility Layer

2016-06-17
A new WinFsp release is available for download from this location

.

The big news in this release is the first availability of a FUSE compatibility layer for WinFsp. Yes, WinFsp now has a FUSE API! The API has been used to port SSHFS to Windows/Cygwin. You can read the full story of how this port was achieved in the

SSHFS Port Case Study

document.

Here are a few screenshots of SSHFS in action! First starting SSHFS from the Cygwin command line.

FUSE for Windows/Cygwin now available

Accessing drive X: from Windows Explorer. Yes, this the home directory on my Macbook Pro.

FUSE for Windows/Cygwin now available

Finally here is the repository for the SSHFS Port.

Tags:Release

SSHFS Port Case Study – Step 4

2016-06-17

This series of articles is a case study in porting SSHFS to Windows and WinFsp. At the time of this writing WinFsp has a native API, but no FUSE compatible API. The main purpose of this case study is to develop a FUSE compatible API for WinFsp.

Step 4: Implementing FUSE Core

We are now finally ready to implement the fuse_operations . This actually proves to be a straightforward mapping of the WinFSP FSP_FILE_SYSTEM_INTERACE to fuse_operations :

GetVolumeInfo

Mapped to statfs . Volume labels are not supported by FUSE (see below).

SetVolumeLabel

No equivalent on FUSE, so simply return STATUS_INVALID_PARAMETER . One thought is to map this call into a setxattr("sys.VolumeLabel") (or similar) call on the root directory ( / ).

GetSecurityByName

Mapped to fgetattr / getattr . The returned stat information is translated into a Windows security descriptor using FspPosixMapPermissionsToSecurityDescriptor .

Create

This is used to create a new file or directory. If a file is created this is mapped to create or mknod ; open . If a directory is created this is mapped to mkdir ; opendir calls (the reason is that on Windows a directory remains open after being created). In some circumstances a chown may be issued as well. After the file or directory has been created a fgetattr / getattr is issued to get stat information to return to the FSD.

Open

This is used to open a new file or directory. First a fgetattr / getattr is issued. If the file is not a directory it is followed by open . If the file is a directory it is followed by opendir .

Overwrite

This is used to overwrite a file when one of the FILE_OVERWRITE , FILE_SUPERSEDE or FILE_OVERWRITE_IF flags has been set. Mapped to ftruncate / truncate .

Cleanup

Mapped to unlink when deleting a file and rmdir when deleting a directory.

Close

Mapped to flush ; release when closing a file and releasedir when closing a directory.

Read

Mapped to read .

Write

Mapped to fgetattr / getattr and write .

Flush

Mapped to fsync or fsyncdir .

GetFileInfo

Mapped to fgetattr / getattr .

SetBasicInfo

Mapped to utimens / utime .

SetAllocationSize

Mapped to fgetattr / getattr followed by ftruncate / truncate . Note that this call and SetFileSize may be consolidated soon in the WinFsp API.

SetFileSize

Mapped to fgetattr / getattr followed by ftruncate / truncate . Note that this call and SetAllocationSize may be consolidated soon in the WinFsp API.

CanDelete

For directories only: mapped to a getdir / readdir call to determine if they are empty and can therefore be deleted.

Rename

Mapped to fgetattr / getattr on the destination file name and rename .

GetSecurity

Mapped to fgetattr / getattr . The returned stat information is translated into a Windows security descriptor using FspPosixMapPermissionsToSecurityDescriptor .

SetSecurity

Mapped to fgetattr / getattr followed by chmod and/or chown .

ReadDirectory

Mapped to getdir / readdir . Note that because of how the Windows directory enumeration API’s work there is a further fgetattr / getattr per file returned!

Some Additional Challenges

Let us now discuss a couple of final challenges in getting a proper FUSE port working under Cygwin: the implementation of fuse_set_signal_handlers / fuse_remove_signal_handlers and fuse_daemonize .

Let us start with fuse_set_signal_handlers / fuse_remove_signal_handlers . Cygwin supports POSIX signals and we can simply set up signal handlers similar to what libfuse does. However this simple approach does not work within WinFsp, because it uses native API’s that Cygwin cannot interrupt with its signal mechanism. For example, the fuse_loop FUSE call eventually results in a WaitForSingleObject API call that Cygwin cannot interrupt. Even trying with an alertable WaitForSingleObjectEx did not work as unfortunately Cygwin does not issue a QueueUserAPC when issuing a signal. So we need an alternative mechanism to support signals.

The alternative is to use sigwait in a separate thread. Fsp_fuse_signal_handler is a WinFsp API that knows how to interrupt that WaitForSingleObject (actually it just signals the waited event).

static inline void *fsp_fuse_signal_thread(void *psigmask) {     int sig;      if (0 == sigwait(psigmask, &sig))         fsp_fuse_signal_handler(sig);      return 0; }

Let us now move to fuse_daemonize . This FUSE call allows a FUSE file system to become a (UNIX) daemon. This is achieved by using the POSIX fork call, which unfortunately has many limitations in Cygwin. One such limitation (and the one that bit us in WinFsp) is that it does not know how to clone Windows heaps ( HeapAlloc / HeapFree ).

Recall that WinFsp uses its own memory allocator (just a thin wrapper around HeapAlloc / HeapFree ). This means that any allocations made prior to the fork() call are doomed after a fork(); with good luck the pointers will point to invalid memory and one will get an Access Violation; with bad luck the pointers will point to valid memory that contains bad data and the program may stumble for a while, just enough to hide the actual cause of the problem.

Luckily there is a rather straightforward work-around: "do not allocate any non-Cygwin resources prior to fork". This is actually possible within WinFsp, because we are already capturing the Cygwin environment and its malloc / free (see fsp_fuse_env in "Step 2"). It is also possible, because the typical FUSE program structure looks like this:

fuse_new fuse_daemonize          // do not allocate any non-Cygwin resources prior to this fuse_loop/fuse_loop_mt  // safe to allocate non-Cygwin resources fuse_destroy

With this change fuse_daemonize works and allows me to declare the Cygwin portion of the SSHFS port complete!

Tags:SSHFS, Port

SSHFS Port Case Study – Step 3

2016-06-07

This series of articles is a case study in porting SSHFS to Windows and WinFsp. At the time of this writing WinFsp has a native API, but no FUSE compatible API. The main purpose of this case study is to develop a FUSE compatible API for WinFsp.

Step 3: Mapping Windows to POSIX

It would seem that we are now ready to start implementing the fuse_operations . However there is another matter that we need to attend to first and that is mapping the Windows file system view of the world to the POSIX one and vice-versa.

Mapping Paths

The Windows and POSIX file systems both use paths to address files. The path conventions are different, so we need a technique to convert between the two. This goes beyond a simple translation of the backslash character ( / ) to slash ( / ), because several characters are reserved and cannot be used in a Windows file path, but are legal when used in a POSIX path.

The reserved Windows characters are:

<   >   :   "   /   /   |   ?   * any character between 0 and 31

POSIX only has two reserved characters: slash ( / ) and NUL .

So how do we map between the two? Luckily this problem has been solved before by "Services for Macintosh" (SFM), "Services for UNIX" (SFU) and Gygwin. The solution involves the use of the Unicode "private use area". When mapping a POSIX path to Windows, if we encounter any of the Windows reserved characters we simply map it to the Unicode range U+F000 – U+F0FF. The reverse mapping from Windows to POSIX is obvious.

Mapping Security

Mapping Windows security to POSIX (and vice-versa) is a much more interesting (and difficult) problem. We have the following requirements:

  • We need a method to map a Windows SID (Security Identifier) to a POSIX uid/gid.

  • We need a method to map a Windows ACL (Access Control List) to a POSIX permission set.

  • We want any mapping method we come up with to be bijective (to the extent that it is possible).

Luckily "Services for UNIX" (and Cygwin) come to the rescue again. The following Cygwin document describes in great detail a method to map a Windows SID to a POSIX uid/gid that is compatible with SFU: https://cygwin.com/cygwin-ug-net/ntsec.html . A different document from SFU describes how to map a Windows ACL to POSIX permissions: https://technet.microsoft.com/en-us/library/bb463216.aspx .

The mappings provided are not perfect, but they come pretty close. They are also proven as they have been used in SFU and Cygwin for years.

WinFsp Implementation

A WinFsp implementation of the above mappings can be found in the file src/dll/posix.c .

Tags:SSHFS, Port

SSHFS Port Case Study – Step 2

2016-06-02

This series of articles is a case study in porting SSHFS to Windows and WinFsp. At the time of this writing WinFsp has a native API, but no FUSE compatible API. The main purpose of this case study is to develop a FUSE compatible API for WinFsp.

Step 2: Create a FUSE Compatible Package

After a few days of development there exists now an initial FUSE implementation within WinFsp. Most of the FUSE API’s from the header files fuse.h , fuse_common.h and fuse_opt.h have been implemented. However none of the fuse_operations currently work as the necessary work to translate WinFsp requests to FUSE requests has not happened yet.

Challenges

  • The FUSE API is old and somewhat hairy. There are multiple versions of it and choosing the right one was not easy. In the end version 2.8 of the API was chosen for implementation.

  • The FUSE API uses a number of OS specific types (notably struct stat ). Sometimes these types have multiple definitions even within the same OS (e.g. struct stat and struct stat64 ). For this reason it was decided to define our own fuse_* types (e.g. struct fuse_stat ) instead of relying on the ones that come with MSVC. Care was taken to ensure that these types remain compatible with Cygwin as it is one of our primary target environments.

  • The WinFsp DLL does not use the MSVCRT and uses its own memory allocator ( HeapAlloc , HeapFree ). Even if it used the MSVCRT malloc , it does not have access to the Cygwin malloc . The FUSE API has a few cases where users are expected to use free to deallocate memory (e.g. fuse_opt_add_opt ). But which free is that for a Cygwin program? The Cygwin free , the MSVCRT free or our own MemFree ?

    To solve this problem we use the following pattern: every FUSE API is implemented as a static inline function that calls a WinFsp-FUSE API and passes it an extra argument that describes the environment:

    static inline int fuse_opt_add_opt(char **opts, const char *opt) {     return fsp_fuse_opt_add_opt(fsp_fuse_env(), opts, opt); }

    The fsp_fuse_env function is another static inline function that simply "captures" the current environment (things like the environment’s malloc and free ).

    ... #elif defined(__CYGWIN__) ... #define FSP_FUSE_ENV_INIT               /     {                                   /         'C',                            /         malloc, free,                   /         fsp_fuse_daemonize,             /         fsp_fuse_set_signal_handlers,   /         fsp_fuse_remove_signal_handlers,/     } ... #else ...  static inline struct fsp_fuse_env *fsp_fuse_env(void) {     static struct fsp_fuse_env env = FSP_FUSE_ENV_INIT;     return &env; }

  • The implementation of fuse_opt proved an unexpected challenge. The function fuse_opt_parse is very flexible, but it also has a lot of quirks. It took a lot of trial and error to arrive at a clean reimplementation.

Things that worked rather nicely

  • The pattern fuse_new / fuse_loop / fuse_destroy fits nicely to the WinFsp service model: FspServiceCreate / FspServiceLoop / FspServiceDelete . This means that every (high-level) FUSE file system can rather easily be converted into a Windows service if desired.

Integrating with Cygwin

It remains to show how to use the WinFsp-FUSE implementation from Cygwin and SSHFS. SSHFS uses pkg-config for its build configuration. Pkg-config requires a fuse.pc file:

arch=x64 prefix=${pcfiledir}/.. incdir=${prefix}/inc/fuse implib=${prefix}/bin/winfsp-${arch}.dll  Name: fuse Description: WinFsp FUSE compatible API Version: 2.8 URL: http://www.secfs.net/winfsp/ Libs: "${implib}" Cflags: -I"${incdir}"

The WinFsp installer has been modified to place this file within its installation directory. It remains to point pkg-config to the appropriate location (using PKG_CONFIG_PATH ) and the SSHFS configuration process can now find the FUSE package.

SSHFS-Win

The sshfs-win open-source project (work in progress) can be found here: https://bitbucket.org/billziss/sshfs-win

Tags:SSHFS, Port

SSHFS Port Case Study – Step 1

2016-05-25

This series of articles is a case study in porting SSHFS to Windows and WinFsp. At the time of this writing WinFsp has a native API, but no FUSE compatible API. The main purpose of this case study is to develop a FUSE compatible API for WinFsp.

Step 1: Gather Information about SSHFS

The SSHFS project is one of the early FUSE projects. The project was originally written by Miklos Szeredi who is also the author of FUSE. SSHFS provides a file system interface on top of SFTP (Secure File Transfer Protocol).

The project’s website is at https://github.com/libfuse/sshfs . A quick perusal of the source code shows that this is a POSIX program, the file configure.ac further shows that it depends on GLib and FUSE.

Luckily Cygwin on Windows provides a POSIX interface and it also includes GLib and pkg-config. We are missing FUSE of course. Let’s try it anyway:

billziss@windows:~/Projects/ext$ git clone https://github.com/libfuse/sshfs.git Cloning into 'sshfs'... [snip] billziss@windows:~/Projects/ext$ cd sshfs/ billziss@windows:~/Projects/ext/sshfs [master]$ autoreconf -i [snip] billziss@windows:~/Projects/ext/sshfs [master]$ ./configure [snip] configure: error: Package requirements (fuse >= 2.3 glib-2.0 gthread-2.0) were not met:  No package 'fuse' found

As expected we get an error because there is no package named FUSE. So let’s create one.

Tags:SSHFS, Port

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » FUSE for Windows/Cygwin now available

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址