GitLocker: The Coding Marketplace

Description:

pyftpdlibustcblog 1.4.1

Quick links

Home
Documentation
Download
Blog
Mailing list
What’s new

About
Python FTP server library provides a high-level portable interface to easily
write very efficient, scalable and asynchronous FTP servers with Python. It is
the most complete RFC-959 FTP server
implementation available for Python programming
language and it’s used in projects like
Google Chromium and
Bazaar and included in
Debian,
Fedora and
FreeBSD package repositories.

Features

Extremely lightweight, fast and scalable (see
why and
benchmarks).
Uses sendfile(2) (see pysendfile)
system call for uploads.
Uses epoll() / kqueue() / select() to handle concurrency asynchronously.
…But can optionally skip to a
multiple thread / process
model (as in: you’ll be free to block or use slow filesystems).
Portable: entirely written in pure Python; works with Python from 2.4 to
3.4 (by using a single code base).
Supports FTPS (RFC-4217),
IPv6 (RFC-2428),
Unicode file names (RFC-2640),
MLSD/MLST commands (RFC-3659).
Support for virtual users and virtual filesystem.
Extremely flexible system of “authorizers” able to manage both “virtual” and
“real” users on on both
UNIX
and
Windows.
Test coverage
close to 100%.

Performances
Depite being written in an intepreted language, pyftpdlib has transfer rates
superior to most common UNIX FTP servers. It also scales better since whereas
vsftpd and proftpd use multiple processes to achieve concurrency, pyftpdlib
will only use one process and handle concurrency asynchronously (see
the C10K problem). Here are some
benchmarks
made against my Linux 3.0.0 box, Intel core-duo 3.1 Ghz:

pyftpdlib vs. proftpd 1.3.4

benchmark type
pyftpdlib
proftpd
speedup

STOR (client -> server)
585.90 MB/sec
600.49 MB/sec
-0.02x

RETR (server -> client)
1652.72 MB/sec
1524.05 MB/sec
+0.08

300 concurrent clients (connect, login)
0.19 secs
9.98 secs
+51x

STOR (1 file with 300 idle clients)
585.59 MB/sec
518.55 MB/sec
+0.1x

RETR (1 file with 300 idle clients)
1497.58 MB/sec
1478.19 MB/sec
0x

300 concurrent clients (RETR 10M file)
3.41 secs
3.60 secs
+0.05x

300 concurrent clients (STOR 10M file)
8.60 secs
11.56 secs
+0.3x

300 concurrent clients (QUIT)
0.03 secs
0.39 secs
+12x

pyftpdlib vs. vsftpd 2.3.5

benchmark type
pyftpdlib
proftpd
speedup

STOR (client -> server)
585.90 MB/sec
611.73 MB/sec
-0.04x

RETR (server -> client)
1652.72 MB/sec
1512.92 MB/sec
+0.09

300 concurrent clients (connect, login)
0.19 secs
20.39 secs
+106x

STOR (1 file with 300 idle clients)
585.59 MB/sec
610.23 MB/sec
-0.04x

RETR (1 file with 300 idle clients)
1497.58 MB/sec
1493.01 MB/sec
0x

300 concurrent clients (RETR 10M file)
3.41 secs
3.67 secs
+0.07x

300 concurrent clients (STOR 10M file)
8.60 secs
9.82 secs
+0.07x

300 concurrent clients (QUIT)
0.03 secs
0.01 secs
+0.14x

For more benchmarks see here.

Quick start
>>> from pyftpdlib.authorizers import DummyAuthorizer
>>> from pyftpdlib.handlers import FTPHandler
>>> from pyftpdlib.servers import FTPServer
>>>
>>> authorizer = DummyAuthorizer()
>>> authorizer.add_user("user", "12345", "/home/giampaolo", perm="elradfmw")
>>> authorizer.add_anonymous("/home/nobody")
>>>
>>> handler = FTPHandler
>>> handler.authorizer = authorizer
>>>
>>> server = FTPServer(("127.0.0.1", 21), handler)
>>> server.serve_forever()
[I 13-02-19 10:55:42] >>> starting FTP server on 127.0.0.1:21 <<<
[I 13-02-19 10:55:42] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-19 10:55:42] masquerade (NAT) address: None
[I 13-02-19 10:55:42] passive ports: None
[I 13-02-19 10:55:42] use sendfile(2): True
[I 13-02-19 10:55:45] 127.0.0.1:34178-[] FTP session opened (connect)
[I 13-02-19 10:55:48] 127.0.0.1:34178-[user] USER 'user' logged in.
[I 13-02-19 10:56:27] 127.0.0.1:34179-[user] RETR /home/giampaolo/.vimrc completed=1 bytes=1700 seconds=0.001
[I 13-02-19 10:56:39] 127.0.0.1:34179-[user] FTP session closed (disconnect).
other code samples

Donate
A lot of time and effort went into making pyftpdlib as it is right now.
If you feel pyftpdlib is useful to you or your business and want to support its
future development please consider donating me
(Giampaolo Rodola’) some money.
I only ask for a small donation, but of course I appreciate any amount.

Don’t want to donate money? Then maybe you could
write me a recommendation on Linkedin.
In case you’re using pyftpdlib into a software of yours
mail me and I’ll add your
software to the
adoptions list.

Timeline

2014-06-03: version 1.4.0 released.
2014-04-12: version 1.3.1 released.
2013-11-07: version 1.3.0 released.
2013-04-22: version 1.2.0 released.
2013-04-09: version 1.1.0 released.
2013-02-22: version 1.0.1 released.
2013-02-19: version 1.0.0 released.
2012-05-14: pyftpdlib included in ftp-cloudfs project.
2012-01-25: version 0.7.0 released.
2011-12-01: pyftpdlib included in feitp-server project.
2011-09-26: pyftpdlib included in ftpmaster project.
2011-07-09: pyftpdlib included in bftpd project.
2011-07-09: pyftpdlib included in fastersync project.
2011-01-31: pyftpdlib included in put.io FTP connector project.
2011-01-24: version 0.6.0 released.
2010-12-14: added donations.
2010-08-24: pyftpdlib included in peerscape project.
2010-07-15: pyftpdlib included in Faetus project.
2010-07-11: pyftpdlib included in Pyfilesystem project.
2010-06-28: pyftpdlib has been packaged for Debian
2010-04-28: pyftpdlib included in sierramodulepos project.
2010-03-20: http://www.smartfile.com uses pyftpdlib.
2010-01-13: pyftpdlib included in zenftp project.
2009-12-26: pyftpdlib included in Symbian Python FTP server project.
2009-11-04: www.netplay.it uses pyftpdlib.
2009-11-04: www.adcast.tv uses pyftpdlib.
2009-11-04: www.bitsontherun.com uses pyftpdlib.
2009-11-02: pyftpdlib included in ftp-cloudfs project.
2009-09-14: version 0.5.2 released.
2009-08-10: pyftpdlib included in Imgserve project.
2009-07-22: pyftpdlib included in Plumi project.
2009-04-02: pyftpdlib RPM-packaged and ported on Fedora to make users can easily install on it via yum install pyftpdlib.
2009-03-28: pyftpdlib included in Bazaar project.
2009-02-23: pyftpdlib included in ShareFTP project.
2009-01-21: version 0.5.1 released.
2008-12-27: pyftpdlib included in Google Chromium, the open source project behind Google Chrome.
2008-12-27: pyftpdlib ported on GNU Darwin systems to make users can easily install on it.
2008-11-26: pyftpdlib included in OpenERP.
2008-10-26: pyftpdlib included in Python for OpenVMS as standard package.
2008-10-09: pyftpdlib included in Shareme project.
2008-09-20: version 0.5.0 released.
2008-08-10: pyftpdlib included in Manent project.
2008-05-16: version 0.4.0 released.
2008-04-09: pyftpdlib used as backend for gpftpd, an FTP server for managing files hosted on Google Pages.
2008-01-17: version 0.3.0 released.
2007-10-14: pyftpdlib included in Aksy project.
2007-09-17: version 0.2.0 released.
2007-09-08: pyftpdlib included as FarManager plug-in.
2007-03-06: pyftpdlib ported on FreeBSD systems to make users can easily install on it.
2007-03-07: version 0.1.1 released.
2007-02-26: version 0.1.0 released.
2006-09-26: initial clunky thread-based progenitor link.

Trademarks
Some famous trademarks which adopted pyftpdlib (complete list).