December 2008
Perl SSL Modules
"You're in a twisty little maze of passages, all different"
-- Will Crowther and Don Woods, "Colossal Cave Adventure"
If you want to implement SSL in perl, there's what seems like a dizzying maze of
available options. Some better and some worse. Some more suited
to one thing, and some to another. This article will help find your
way through some of that maze (but not all of it). A thorough review
would be nice, but this is not that.
I set out to write an IMAP mail client in perl, and with that in mind,
I won't have much to say about SSL modules that are only or primarily
concerned with the secure web (https) protocol. My goal was to come up
with an implementation of a program that would work on the most different
platforms, without too much trouble on my part, and without assuming that
end-users can install perl modules.
I had to learn an unncessary amount about too many implementations none
of which work extremely well.
It's all a good argument for having some sort of perl-poobah that would
oversee module development and bless certain recommended interfaces.
At any rate, for IMAP, I wanted the ability to establish a secure
connection, and I wanted to do non-blocking reads from the socket
descriptor from within a polling (select()) loop. I also wanted
the ability to take an existing connection, and start up the SSL
stuff in the middle, to support the IMAP STARTTLS command on an
insecure connection.
I will briefly say that for the web in most cases, just use LWP.
It will work as long as you have an appropriate SSL module installed.
I can't give a concrete answer on what "appropriate" means, but I'm sure
that Net::SSL (which is in turn part of Crypt::SSLeay) works. I've seen
it reported that
Net::SSLeay (and maybe also IO::Socket::SSL) will work, but that doesn't
match with my testing. Then again, my error message wasn't the
expected one of needing a module, so maybe it's my mistake, or
maybe my LWP isn't up-to-date.
Overview
There are two main modules that have everything needed to set up
an SSL connection. These are Crypt::SSLeay, and Net::SSLeay. Of the
two, Crypt::SSLeay seems to be a bit more stable for my purposes
than Net::SSLeay.
It is hopefully possible to write anything you need with either of these
modules. However, they don't provide a lot of convenience or quality
documentation for this. Because of this, there's other included or add-on
modules that attempt to simplify things.
The following modules are the things that I'm aware of that can do what
we need:
Crypt::SSLeay
Net::SSL
Net::SSLeay
Net::SSLeay::Handle
IO::Socket::SSL
Net::Server::Proto::SSL
For my purposes, IO::Socket::SSL provides the most features I wanted,
and Net::SSL close behind. They'll reverse positions if I can switch a live
connection to SSL with Net::SSL, becauase Net::SSL looks to be better
maintained and I think it deals with file handles better..
IO::Socket::SSL lives on top of Net::SSLeay. On the outside Net::SSLeay is
primarily oriented towards https, but it supposedly has lots of
good stuff hidden inside (and hidden from documentation too). You can
use Net::SSLeay::Handle
for non web things, but dealing with it's file descriptors is at least
a pain, and possibly completely broken. Net::SSLeay itself can supposedly
do non-web things, but it isn't well documented. Net::Server::Proto::SSL
is so far "experimental". It's somehow related to IO::Socket::SSL.
Crypt::SSLeay and Net::SSL
This is where I started. Net::SSL is part of Crypt::SSLeay. Starting
up an SSL connection is pretty straightforward:
use Net::SSL;
...
$sock = Net::SSL->new(PeerAddr => $host, PeerPort => $port,
Proto => 'tcp', Timeout => 200) || die "sslsocket";
...
print $sock->getchunk;
Net::SSL includes a getchunk() function, which just grabs all the data
available on the network (up to 32K) and gives it to you. It blocks
until something is read, or a timeout expires. This worked great
for my purposes.
Since I was using an IMAP server that was only running the secure
port, this was also fine for testing. But if there's a way in
Net::SSL to convert an existing connection to SSL, it's not obvious
from the documentation.
Net::SSLeay::Handle
But, alas, I wanted to test things on a Mac, and Leopard does not have
Net::SSL. But it does have Net::SSLeay (go figure). So my first attempt
to find something that would work on that platform was Net::SSLeay::Handle.
It looked a little less straight-forward, using the semi-obscure tie
command, but still pretty simple. And it looks like it should support
converting a regular socket connection into an SSL connection midstream.
use Net::SSLeay::Handle;
...
tie(*S2,"Net::SSLeay::Handle", $host, $port);
my $sock = \*S2;
So that seemed to work. At least, it made a connection, and reading from
<$sock> would do just what you'd hope. But things went downhill from
there. I didn't want to use <$sock>, I wanted to use read(), to read a
chunk of data from the socket, just the way getchunk() in Net::SSL works.
In other words something like "read($sock,$buf,8192);".
Now this should be fine - with normal socket reading semantics, a read
will block until it gets something, but then it will return even if
it doesn't get everything. Perl does get in the way here a bit, but
with normal perl sockets, using sysread() skips all that rot, and sysread()
from a socket works just like I'd expect.
But this didn't do that. Using either read or sysread, the call blocked
until the buffer size I gave was satisfied. Not good. Fine, I thought
I can still work with this, I'll just make the socket completely non-blocking
with "$sock->blocking(0);" (hint: don't forget to add in "use IO::Handle;").
That sort-of worked. But when I tried to put things inside of a select(),
I found that $sock was getting select()ed, but then a subsequent sysread
would find nothing there to read. At this point I punted. Granted,
the error may have been mine, and I could have been minutes away from solving
it, but I was getting annoyed.
IO::Socket::SSL
So, I tried IO::Socket::SSL, also available in a default Leopard
installation.
use IO::Socket::SSL;
...
$sock = IO::Socket::SSL->new("$host:$port");
...
sysread($sock,$buf,8192);
Happy happy, joy joy. It worked fine. sysread() does exactly what I
expect it to, as does select(). I don't care for the "$host:$port"
syntax (too web-specific looking), but that's a minor quibble.
But that's still not a perfect solution.
A merged approach
The thing that doesn't work in IO::Socket::SSL is the $sock->read().
This has the same problem as the read command does in Net::SSLeay::Handle,
where it blocks until it reads the full amount. The reason this would
make me happy is that even though I started with getchunk() in Net::SSL,
$sock->read() also does pretty much what I want. So I have $sock->read()
available in both modules, but unfortunately, they don't behave the same,
and I haven't found any easy way to make them.
Nevertheless, I needed a solution that would let one program run on
platforms which had different SSL options. I can at least cover what
seem to be the two most popular approaches.
So here's what I've got at this point. If there's a better way, I'd
be happy to hear it.
eval {
require Net::SSL;
Net::SSL->import();
};
if ($@) {
require IO::Socket::SSL;
IO::Socket::SSL->import();
if ($@) {
print STDERR "Can't find SSL module!\n";
exit(-1);
}
$SSL="Net";
} else {
$SSL="Crypt";
}
. . .
if ($SSL eq "Crypt") {
$sock = Net::SSL->new(PeerAddr => $host, PeerPort => $port,
Proto => 'tcp', Timeout => 200) || die "new Net::SSL";
} else {
$sock = IO::Socket::SSL->new("$host:$port") || die "new IO::Socket::SSL";
}
. . .
sub GetChunk {
my $s=shift(@_);
if ($SSL eq "Crypt") {
return($s->getchunk);
} else {
my $buf="";
sysread($s,$buf,8192);
return($buf);
}
}
And, I'm still working on implementing STARTTLS, which requires
jump-starting SSL on an already in-use socket. I'm pretty sure I can
do it with IO::Socket::SSL. I'll see if I can make Net::SSL do it too.
Reader Comments
(Experimental. Moderated, expect delays. Posts may be edited or ignored.
I reserve the right to remove any or all comments, at any time.)
6 comments:
At 2008/12/20 13:23
Scott Gifford wrote:
|
Have you tried making the sockets nonblocking, like this: $sock->blocking(0); For me at least, that seems to give the effect you're looking for.
|
At 2008/12/20 13:34
 wrote:
|
Yes, I've tried that. It sort of works, but the behaviour is subtly different. Standard socket reading behaviour is to block until it reads something. The bad behavior I was getting was to block until it reads everything that was requested. With your non-blocking suggestion, it's possible to return after reading nothing. A select() would in theory eliminate this issue, but when I tried non-blocking, I wasn't getting correct behavior from the select() -- it would come up in the select() when nothing was there to be read; though admittedly I didn't try to thoroughly debug that issue. tom
|
At 2008/12/20 13:59
Scott Gifford wrote:
|
Ah, I see. Select and SSL don't always mix well, but I have had pretty good luck using nonblocking sockets with select() and sysread(), just ignoring errors EAGAIN and EINTR returned by sysread().
|
At 2010/01/09 20:04
Leif Pedersen wrote:
|
I'm navigating this chaos, and it seems that select() with SSL is supported (you would think Lighttpd proves that!). Anyway, you have to avoid some assumptions. Do not assume a read call only reads or a write call only writes. Do not assume reading zero bytes means EOF. A read may return zero if it only processed the beginning of a header but no application data, for example. A read call may need to write, and a write call may need to read because either side may initiate a new handshake midstream. So be sure to set O_NONBLOCK so that you don't accidentally block for write when you call read. Also pay attention to the error codes, which will indicate whether it needs to read or write next, and update select()'s fdset to watch for what SSL needs. Many iterations may be necessary before the application's data stream progresses. I found this in docs about the underlying C libraries: http://www.openssl.org/support/faq.html#PROG10 and http://www.openssl.org/docs/ssl/SSL_get_error.html That said, I haven't actually gotten anything cobbled together yet. I'm writing something that is a single thread and is absolutely never allowed to block unless there's actually nothing to do. I'm trying to stick with the lowest-level libraries I can because the more wrappers there are the more likely it is that I'll trip over some bug or limitation. Often those wrappers are far more complicated than necessary, but sometimes they really do a lot of necessary work for you. Have you had any more recent findings?
|
At 2010/01/10 19:03
Leif Pedersen wrote:
|
I finally finished putting the pieces together, and wrote up my solution in a longer article. Thanks for sharing your findings, they saved me some time. Here are my findings: http://devpit.org/wiki/OpenSSL_with_nonblocking_sockets_(in_Perl)
|
At 2012/02/14 10:52
wrote:
|
How do you use IO::Socket::SSL in a client software to connect to a server through a proxy? Searching through the web, I read somewhere that NET::SSL provides the support for proxy, but not IO::Socket::SSL. Do you know or had any experience on this? Thanks,
|
End Comments
Add a comment
