curlftpfs doesn't work but 'ftp' yes

uzsolt

uzsolt

I've an access to an FTP server. I can use it with ftp() but can't with sysutils/fusefs-curlftpfs. It mounted but doesn't show anything (files) and I can't touch foo ("operation not supported") on mounted directory.

I don't know what other informations/outputs need.

So the question: how can I mount this FTP-server?
 
Short answer, don't use sysutils/fusefs-curlftpfs from the ports which is based on curlftpfs-0.9.2 but download/unpack/configure/make/install curlftpfs-0.9.1. Uninstall curlftpfs-0.9.2 but not sysutils/fusefs-libs.

curl -LO https://curlftpfs.googlecode.com/files/curlftpfs-0.9.1.tar.gz
tar -xzf curlftpfs-0.9.1.tar.gz
cd curlftpfs-0.9.1
configure
sed -e 's/LDFLAGS =/LDFLAGS =-liconv/' -i "" Makefile
make install clean

The long story is that the original author of curlftpfs Robson Braga Araujo obviously left the project in the well working state 0.9.1. Another submitter jumped in, and switched the working buffered file uploading to threaded file uploading, and this was published as 0.9.2, i.e. the version on which our port is based on.

It looks like said submitter tried to implement this by the way of a Producer–Consumer Mechanism. However he utilized in total 4 independent semaphores for thread synchronization. Without in deep analysis it is pretty clear that with so many semaphores there is a high possibility of thread dead-locking in the course of file uploads, and this is actually happening on FreeBSD and Mac OS X.

So the solution is to switch back to the very nice original code of Robson Braga Araujo, i.e. curlftpfs-0.9.1, and simple dump the additions of the other submitter. Did these really ever work with Linux?

I wrote a BLog post in German language on adapting curlftps for Mac OS X. I started with curlftpfs-0.9.2, only to learn that this is seriously broken. I found that it is broken also on FreeBSD, and then I based all my further work on the non-spoiled original code. For Mac OS X, I added also another caching mechanism to let the Finder display directory contents more quickly. Most probably this addition is not needed on FreeBSD.

http://blog.obsigna.net/?p=529
 
Are these the steps you followed?

$ mkdir myFTP
$ curlftpfs ftp://ftp.myFTP.net/ myFTP/
$ cd myFTP/
$ ls -l
total 0

//CONTENTS LISTED HERE...

$ cd ..
$ fusermount -u myFTP/
 
Next thing you need to do is to repeat the commands while in debug mode so you can get useful info (and post them here):

curlftpfs -d -v -o ftpfs_debug ftp://ftp.myFTP.net/ myFTP/

Then
cd myFTP && ls -l
 
Code: 
Script started on Wed Feb  4 08:44:21 2015  [138/7624]
command: curlftpfs -d -v SERVER ftp/
*  Trying SERVER-IP...
* Connected to SERVER (SERVER-IP) port 21 (#0)
< 220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
< 220-You are user number 1 of 50 allowed.
< 220-Local time is now 08:44. Server port: 21.
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 15 minutes of inactivity.
> USER USERNAME
< 331 User USERNAME OK. Password required
> PASS PASSWORD
< 230 OK. Current restricted directory is /
> PWD
< 257 "/" is your current location
* Entry path is '/'
* ftp_perform ends with SECONDARY: 0
* Remembering we are in dir ""
* Connection #0 to host SERVER left intact
FUSE library version: 2.9.3
nullpath_ok: 0
nopath: 0
utime_omit_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56, pid: 13001
INIT: 7.8
flags=0x00000000
max_readahead=0x00010000
  INIT: 7.19
  flags=0x00000010
  max_readahead=0x00010000
  max_write=0x00020000
  max_background=0
  congestion_threshold=0
  unique: 1, success, outsize: 40
unique: 2, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13036
getattr /
* Found bundle for host SERVER: 0x8044464a0
* Re-using existing connection! (#0) with host SERVER
* Connected to SERVER (SERVER-IP) port 21 (#0)
* Request has same path as previous transfer
> PASV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 227 Entering Passive Mode (SERVER-IP,124,133)
*  Trying SERVER-IP...
* Connecting to SERVER-IP (SERVER-IP) port 31877
* Connected to SERVER (SERVER-IP) port 21 (#0)
> TYPE A
< 200 TYPE is now ASCII
> LIST -a
< 150 Accepted data connection
* Maxdownload = -1
* Remembering we are in dir ""
< 226-Options: -a -l
< 226 4 matches total
* Connection #0 to host SERVER left intact
  unique: 2, success, outsize: 112
unique: 3, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13002
getattr /
  unique: 3, success, outsize: 112
unique: 4, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13045
getattr /
  unique: 4, success, outsize: 112
unique: 5, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 13045
  unique: 5, success, outsize: 32
unique: 6, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 13045
statfs /
  unique: 6, success, outsize: 96
unique: 7, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13045
getattr /
  unique: 7, success, outsize: 112
unique: 8, opcode: READDIR (28), nodeid: 1, insize: 64, pid: 13045
getdir[0]
* Found bundle for host SERVER: 0x8044464a0
* Re-using existing connection! (#0) with host SERVER
* Connected to SERVER (SERVER-IP) port 21 (#0)
* Request has same path as previous transfer
> PASV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 227 Entering Passive Mode (SERVER-IP,120,85)
*  Trying SERVER-IP...
* Connecting to SERVER-IP (SERVER-IP) port 30805
* Connected to SERVER (SERVER-IP) port 21 (#0)
> LIST -a
< 150 Accepted data connection
* Maxdownload = -1
* Remembering we are in dir ""
< 226-Options: -a -l
< 226 4 matches total
* Connection #0 to host SERVER left intact
  unique: 8, success, outsize: 16
unique: 9, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 13045
  unique: 9, success, outsize: 16
unique: 10, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 13045
  unique: 10, success, outsize: 32
unique: 11, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 13045
statfs /
  unique: 11, success, outsize: 96
unique: 12, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13045
getattr /
  unique: 12, success, outsize: 112
unique: 13, opcode: READDIR (28), nodeid: 1, insize: 64, pid: 13045
getdir[0]
  unique: 13, success, outsize: 16
unique: 14, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 13045
  unique: 14, success, outsize: 16
unique: 15, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13002
getattr /
* Found bundle for host SERVER: 0x8044464a0
* Re-using existing connection! (#0) with host SERVER
* Connected to SERVER (SERVER-IP) port 21 (#0)
* Request has same path as previous transfer
> PASV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 227 Entering Passive Mode (SERVER-IP,118,45)
*  Trying SERVER-IP...
* Connecting to SERVER-IP (SERVER-IP) port 30253
* Connected to SERVER (SERVER-IP) port 21 (#0)
> LIST -a
< 150 Accepted data connection
* Maxdownload = -1
* Remembering we are in dir ""
< 226-Options: -a -l
< 226 4 matches total
* Connection #0 to host SERVER left intact
  unique: 15, success, outsize: 112
unique: 16, opcode: LOOKUP (1), nodeid: 1, insize: 42, pid: 13058
LOOKUP /x
getattr /x
* Found bundle for host SERVER: 0x8044464a0
* Re-using existing connection! (#0) with host SERVER
* Connected to SERVER (SERVER-IP) port 21 (#0)
* Request has same path as previous transfer
> PASV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 227 Entering Passive Mode (SERVER-IP,128,33)
*  Trying SERVER-IP...
* Connecting to SERVER-IP (SERVER-IP) port 32801
* Connected to SERVER (SERVER-IP) port 21 (#0)
> LIST -a
< 150 Accepted data connection
* Maxdownload = -1
* Remembering we are in dir ""
< 226-Options: -a -l
< 226 4 matches total
* Connection #0 to host SERVER left intact
ftpfs: operation ftpfs_getattr failed because No such file or directory
  unique: 16, error: -2 (No such file or directory), outsize: 16
unique: 17, opcode: LOOKUP (1), nodeid: 1, insize: 42, pid: 13058
LOOKUP /x
getattr /x
  unique: 17, error: -2 (No such file or directory), outsize: 16
unique: 18, opcode: CREATE (35), nodeid: 1, insize: 50, pid: 13058
create flags: 0x202 /x 0100644 umask=0000
ftpfs: operation ftpfs_open failed because Operation not supported
  unique: 18, error: -45 (Operation not supported), outsize: 16
unique: 19, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13002
getattr /
  unique: 19, success, outsize: 112
unique: 20, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 13072
statfs /
  unique: 20, success, outsize: 96
unique: 21, opcode: GETATTR (3), nodeid: 1, insize: 40, pid: 13072
getattr /
  unique: 21, success, outsize: 112
unique: 22, opcode: FORGET (2), nodeid: 1, insize: 48, pid: 13072
FORGET 1/1
unique: 23, opcode: DESTROY (38), nodeid: 0, insize: 40, pid: 13072
  unique: 23, success, outsize: 16

Script done on Wed Feb  4 08:45:02 2015

I tried ls ftp and touch ftp/x.
 
I am assuming FUSE is loaded. Does kldstat -m fuse show it is? If not you should kldload fuse as root and add it to the list of kernel modules that get loaded at reboot with sysrc kld_list="`sysrc -ni kld_list` fuse".
 
curlftpfs-0.9.2 on which sysutils/fusefs-curlftpfs is based on is broken. I tried this one to get it to run on Mac OS X, and I was not able to create/upload files to the remote ftp server -- exactly the same issue as yours.

curlftpfs-0.9.1 is the original version by Robson Braga Araujo, and with only a few tweaks this one can be made working on FreeBSD and on Mac OS X. In the course from 0.9.1 to 0.9.2 another submitter jumped in to the project, and he replaced the original buffered file uploading by a threaded uploading method. For this he implemented a Producer–Consumer mechanism, and he introduced 4 independent semaphores for thread synchronization. The problem with that is, that it is extremely hard to prevent race conditions and thread deadlocking with that many semaphores, and as a matter of fact, deadlocking happens on FreeBSD and Mac OS X upon file uploading (creation and copying).

I gave up trying to fix the threaded uploading method, and instead switched back to the unspoiled original version 0.9.1.

On FreeBSD you would uninstall sysutils/fusefs-curlftpfs but leave sysutils/fusefs-libs in place.

Download the attached patch file curlftpfs_patch.txt, and then:
curl -LO https://curlftpfs.googlecode.com/files/curlftpfs-0.9.1.tar.gz
tar -oxzf curlftpfs-0.9.1.tar.gz
cd curlftpfs-0.9.1
./configure
patch < /path/to/the/downloaded/curlftpfs_patch.txt
make
sudo make install clean

If the fuse kernel module is not already loaded then submit the following command.
kldload fuse.ko

Consider to add the entry fuse_load="YES" to the file /boot/loader.conf and with that, said module will be loaded automatically upon each reboot.

Finally, run some tests:
curlftpfs -o user=<USERNAME>:<PASSWORD>,allow_root ftp.example.com/ /mnt
cd /mnt
ls -l
touch test.txt
ls -l
cp /etc/motd ./
ls -l
 

Attachments

youngunix said:
If you are referring to the content in post#7, then those instructions are under the UAYOR License (Use At Your Own Risk).
Click to expand...

You missed the Unique Selling Point of post#7. The USP is not UAYOR -- this is already part of the original license GPLv2:
GPLv2 - Chapter 11 - NO WARRANTY said:
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
Click to expand...

The USP is: My instructions do work, while the software from the ports does not.
 
If I reply with "No, you misunderstood...blah blah!", you and I will be stuck in an infinite loop which will cause a thread-overflow.
But I will clarify that UAYOR is directed at instructions posted by forum users, NOT at the software used.
Example.
 
You must log in or register to reply here.
Back
Top