Using libcurl

Overview

This is a short overview on using libcurl in your C programs. There are specific MAN pages for each function mentioned. There are also the libcurl-easy MAN page, the libcurl-multi MAN page, the libcurl-share MAN page and the libcurl-tutorial(3) MAN page for in-depth understanding on how to program with libcurl.

libcurl has a global constant environment that is setup and maintained while using libcurl. Call curl_global_init at the start of your program and curl_global_cleanup at the end. For information, see Global Constants below.

To transfer files, always set up an "easy handle" using curl_easy_init. When you want the file(s) transferred, you have the option of using the "easy" interface, or the "multi" interface.

The multi interface is an asynchronous interface that you call and performs only a small part of the transfer on each invoke. It is well-suited if you want to perform actions while the transfer is in progress, or similar. The multi interface allows you to select on libcurl action, and download multiple files simultaneously using a single thread. For more information, see the libcurl-multi MAN page.

You can have multiple easy handles share certain data, even if they are used in different threads. This is accomplished by using the share interface, as described in the libcurl-share MAN page.

There are also other helpful functions to use, including:

Threads

Do not call curl-functions simultaneously using the same handle from several threads. libcurl is thread-safe, and can be used in any number of threads, but use separate curl handles if you want to use libcurl in more than one thread simultaneously.

The global environment functions are not thread-safe. For information, see Global Constants below.

Persistent Connections

Persistent connections mean that libcurl can reuse the same connection for several transfers, if conditions are correct.

libcurl will always attempt to use persistent connections. Whenever you use curl_easy_perform or curl_multi_perform, libcurl will attempt to use an existing connection to perform the transfer, and if none exists, it will open a new one that will be subject for reuse on a possible following call to curl_easy_perform or curl_multi_perform.

To allow libcurl to take full advantage of persistent connections, you should perform as many of your file transfers as possible using the same curl handle. When you call curl_easy_cleanup, all the possible open connections held by libcurl will be closed and forgotten.

NOTE:
The options set with curl_easy_setopt will be used on every repeated curl_easy_perform call.

Global Constants

There are a variety of constants that libcurl uses, mainly through its internal use of other libraries, which are too complicated for the library loader to setup. A program must call a library function, after the program is loaded and running, to finish setting up the library code.

curl_global_init is the function that to call. This may allocate resources and so the companion function curl_global_cleanup releases them.

The basic rule for constructing a program that uses libcurl is this: Call curl_global_init, with a CURL_GLOBAL_ALL argument, immediately after the program starts, while it is still only one thread and before it uses libcurl. Call curl_global_cleanup immediately before the program exits, when the program is again only one thread and after its last use of libcurl.

You can call both of these multiple times, as long as all calls meet these requirements and the number of calls to each is the same.

It is not required that the functions be called at the beginning and end of the program. That is usually the easiest way to do it. It is required that the functions be called when no other thread in the program is running.

These global constant functions are not thread-safe, so do not call them when any other thread in the program is running. It is not adequate that no other thread is using libcurl at the time, because these functions internally call similar functions of other libraries, and those functions are similarly not thread-safe. You cannot know what these libraries are, or whether other threads are using them.

The global constant situation merits special consideration when the code you are writing to use libcurl is not the main program, but rather a modular piece of a program, e.g. another library. As a module, your code does not know about other parts of the program. It does not know whether they use libcurl or not. And its code does not necessarily run at the start and end of the whole program.

A module like this must have global constant functions of its own, just like curl_global_init and curl_global_cleanup. The module has control at the beginning and end of the program and has a place to call the libcurl functions. Note that if multiple modules in the program use libcurl, they all will separately call the libcurl functions. This is acceptable because only the first curl_global_init and the last curl_global_cleanup in a program change anything. (libcurl uses a reference count in static memory).

In a C++ module, it is common to deal with the global constant situation by defining a special class that represents the global constant environment of the module. A program always has exactly one object of the class in static storage. That way, the program automatically calls the constructor of the object as the program starts up and the destructor as it terminates. As the author of this libcurl-using module, you can make the constructor call curl_global_init and the destructor call curl_global_cleanup and satisfy libcurl's requirements without your user having to be concerned about it.

curl_global_init has an argument that tells what particular parts of the global constant environment to setup. To successfully use any value except CURL_GLOBAL_ALL (sets up all), requires specific knowledge of the internal workings of libcurl and all other parts of the program of which it is part.

A special part of the global constant environment is the identity of the memory allocator. curl_global_init selects the system default memory allocator, but you can use curl_global_init_mem to supply one of your own. However, there is no way to use curl_global_init_mem in a modular program; all modules in the program that might use libcurl would have to agree on one allocator.

There is a failsafe in libcurl that makes it usable in basic situations without having to be concerned about the global constant environment: curl_easy_init sets up the environment itself if it has not already been performed. The resources it acquires to do so get automatically released by the operating system when the program exits.

This failsafe feature exists mainly for backward compatibility because previously global functions did not exist. Because the failsafe feature is sufficient only in the most basic of programs, it is not recommended for a program to rely on it.

See Also

Network Overview
HTTP Client Library (libcurl)

Revision History

2013/09/12 Converted.


CONFIDENTIAL