Serverless Gems libcurl

We are experimenting with typhoeus and is has native dependency for libcurl.
Unfortunately, the compiled required libraries (such as libcurl.so and libnghttp2.so) are not been uploaded to the gem layer.
I am not sure, but I guess that Serverless Gems is not aware of such dependency. The deploy process does not report any error though, while executing the lambda function gives me the expected error: lib not found.
Is there a way to tell Serverless that we need some native libraries?

It was a tedious manual process. but I managed to circumvent the issue by creating and using a custom layer with the required libs.

It would be nice if there was an automatic way to add these dependencies, even if just by listing the libs to be added using apt-get.

Bummer. Jets already does native gem detection as part of the deployment, but some edge cases like this will take some time also to detect properly. Glad you were able to use a custom layer to fix this, though. That’s precisely why custom layers are available. Though they are some effort, it’s an option.

Dug into this a little bit and found that it actually leads to the ffi gem that seems to dynamically loads libcurl.so https://github.com/ffi/ffi/blob/master/lib/ffi/library.rb#L98 Most native gems let you know if it’s using native headers. Though it may not sound like it at the surface, this one is quite unique. It’ll take some time to dig further. The goal is to make Serverless Gems seamless, so end-users don’t have to do anything. Over the years, there have been many edge cases that have been handled. Think there will always continue to be some special cases, though. Over time they should dwindle. Thanks for the report, and glad you were able to resolve the issue again.

One thing I’d really like to see is the ability to just specify a Dockerfile containing dependent libraries. Maybe Alpine Linux so we get statically compiled MUSL that will work in Amazon Linux… though maybe docker already has Amazon Linux support?

@roque86 Would you be willing to share some more details (maybe a config file) regarding your workaround. I’m having a similar issue with native library loads (libcurl, etc.), and am on the custom lambda layer path, but struggling with getting it all working. Thanks!

Hi @kevinr.

Sorry for this late reply but I only saw the message now.
I am sure you already solved your problem and to be quite honest, I do not even remember all the steps.
What I do know is that it was basically a “try and error” approach. Build layer, deploy, check lambdas, account for the errors, repeat the process.

@tung Do you know if this gem ever got support added to serverlessgems.com? I’m running into the same issue and seems to be a rather uncommon thing to support.

Figured it out. This one was particularly tricky.

Both ethon and typhoeus should automatically be detected by the serverlessgems service now and pre-compiled versions are available.

Also note: Tried to somewhat future proof this for other gems. Ended up releasing https://github.com/boltops-tools/libcurl Though, you do not have to add the libcurl gem to your Gemfile for ethon and typhoeus gems. They are automatically detected as needing pre-compiled gems.