add curlver.h include to umbrella header to ensure version checks work

[toffaletti]

attempting to fix #5157

I believe the issue is that the version checks here are failing:

swift-corelibs-foundation/Sources/_CFURLSessionInterface/include/CFURLSessionInterface.h

Lines 39 to 43 in c466923

	#if LIBCURL_VERSION_MAJOR > 7 || (LIBCURL_VERSION_MAJOR == 7 && LIBCURL_VERSION_MINOR > 84) || (LIBCURL_VERSION_MAJOR == 7 && LIBCURL_VERSION_MINOR == 84 && LIBCURL_VERSION_PATCH >= 0)
	#define NS_CURL_CURLINFO_CAINFO_SUPPORTED 1
	#else
	#define NS_CURL_CURLINFO_CAINFO_SUPPORTED 0
	#endif

because LIBCURL_VERSION_MAJOR etc have not been defined.
because <curl/curlver.h> hasn't been included anywhere yet.
It is included here, but after CFURLSessionInterface.h has already been processed.

swift-corelibs-foundation/Sources/_CFURLSessionInterface/CFURLSessionInterface.c

Lines 22 to 24 in c466923

	#include "CFURLSessionInterface.h"
	#include "CFString.h"
	#include <curl/curl.h>

swift-corelibs-foundation/Sources/_CFURLSessionInterface/include/static-module.map

Line 2 in c466923

umbrella header "CFURLSessionInterface.h"

[parkera]

Let us know if this works for you.

[toffaletti]

Let us know if this works for you.

I might need your help in understanding how to test this. I haven't built a static sdk from scratch.

[toffaletti]

I was able to build my own static sdk.

with this change:

swift build --swift-sdk=swift-6.0-DEVELOPMENT-SNAPSHOT-2024-12-03-a-17-g9090bb315e0_static-linux-0.0.1

$ swift-contents-of https://www.apple.com
loaded 187307 bytes from https://www.apple.com

without this change:

swift build --swift-sdk swift-6.0.3-RELEASE_static-linux-0.0.1

$ swift-contents-of https://www.apple.com
Error: The operation could not be completed. (NSCocoaErrorDomain error 256.)

[parkera]

@swift-ci test

[toffaletti]

Looks like I've broken the Windows build:

C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-corelibs-foundation/Sources/_CFURLSessionInterface/include\CFURLSessionInterface.h:37:10: error: 'curl/curlver.h' file not found
 35 | #include <winsock2.h>
 36 | #endif
 37 | #include <curl/curlver.h>
    |          `- error: 'curl/curlver.h' file not found

I don't know enough about the windows builders to understand why it is missing, though it doesn't seem like it should be based on Daniel's reaction to here: https://curl.se/mail/archive-2005-01/0089.html

[toffaletti]

Looks like CI is building its own copy of curl 8.9.1:

[2025-02-03 19:37:11] Building 'C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\curl' to 'T:\x64\Windows\curl-8.9.1' for arch 'x86_64'...
cmake.exe -B T:\x64\Windows\curl-8.9.1 -S C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\curl -G Ninja -D BUILD_CURL_EXE=NO -D BUILD_LIBCURL_DOCS=NO -D BUILD_MISC_DOCS=NO -D BUILD_SHARED_LIBS=NO -D BUILD_TESTING=NO -D CMAKE_BUILD_TYPE=Release -D CMAKE_C_COMPILER=cl -D CMAKE_C_FLAGS=/GS- /Gw /Gy /Oi /Oy /Zc:inline -D CMAKE_INSTALL_LIBDIR=lib/Windows/x86_64 -D CMAKE_INSTALL_PREFIX=T:/Library/curl-8.9.1/usr -D CMAKE_POSITION_INDEPENDENT_CODE=YES -D CMAKE_SYSTEM_NAME=Windows -D CURL_BROTLI=NO -D CURL_CA_BUNDLE=none -D CURL_CA_FALLBACK=NO -D CURL_CA_PATH=none -D CURL_DISABLE_ALTSVC=NO -D CURL_DISABLE_AWS=YES -D CURL_DISABLE_BASIC_AUTH=NO -D CURL_DISABLE_BEARER_AUTH=NO -D CURL_DISABLE_BINDLOCAL=NO -D CURL_DISABLE_COOKIES=NO -D CURL_DISABLE_DICT=YES -D CURL_DISABLE_DIGEST_AUTH=NO -D CURL_DISABLE_DOH=NO -D CURL_DISABLE_FILE=YES -D CURL_DISABLE_FORM_API=NO -D CURL_DISABLE_FTP=YES -D CURL_DISABLE_GETOPTIONS=NO -D CURL_DISABLE_GOPHER=YES -D CURL_DISABLE_HEADERS_API=YES -D CURL_DISABLE_HSTS=NO -D CURL_DISABLE_HTTP=NO -D CURL_DISABLE_HTTP_AUTH=NO -D CURL_DISABLE_IMAP=YES -D CURL_DISABLE_KERBEROS_AUTH=NO -D CURL_DISABLE_LDAP=YES -D CURL_DISABLE_LDAPS=YES -D CURL_DISABLE_MIME=NO -D CURL_DISABLE_MQTT=YES -D CURL_DISABLE_NEGOTIATE_AUTH=NO -D CURL_DISABLE_NETRC=NO -D CURL_DISABLE_NTLM=NO -D CURL_DISABLE_PARSEDATE=NO -D CURL_DISABLE_POP3=YES -D CURL_DISABLE_PROGRESS_METER=YES -D CURL_DISABLE_PROXY=NO -D CURL_DISABLE_RTSP=YES -D CURL_DISABLE_SHUFFLE_DNS=YES -D CURL_DISABLE_SMB=YES -D CURL_DISABLE_SMTP=YES -D CURL_DISABLE_SOCKETPAIR=YES -D CURL_DISABLE_SRP=NO -D CURL_DISABLE_TELNET=YES -D CURL_DISABLE_TFTP=YES -D CURL_DISABLE_VERBOSE_STRINGS=NO -D CURL_LTO=NO -D CURL_USE_BEARSSL=NO -D CURL_USE_GNUTLS=NO -D CURL_USE_GSSAPI=NO -D CURL_USE_LIBPSL=NO -D CURL_USE_LIBSSH=NO -D CURL_USE_LIBSSH2=NO -D CURL_USE_MBEDTLS=NO -D CURL_USE_OPENSSL=NO -D CURL_USE_SCHANNEL=YES -D CURL_USE_WOLFSSL=NO -D CURL_WINDOWS_SSPI=YES -D CURL_ZLIB=YES -D CURL_ZSTD=NO -D ENABLE_ARES=NO -D ENABLE_CURL_MANUAL=NO -D ENABLE_CURLDEBUG=NO -D ENABLE_DEBUG=NO -D ENABLE_IPV6=YES -D ENABLE_THREADED_RESOLVER=NO -D ENABLE_UNICODE=YES -D ENABLE_UNIX_SOCKETS=NO -D ENABLE_WEBSOCKETS=YES -D HAVE_POLL_FINE=NO -D USE_ECH=NO -D USE_HTTPSRR=NO -D USE_IDN2=NO -D USE_MSH3=NO -D USE_NGHTTP2=NO -D USE_NGTCP2=NO -D USE_OPENSSL_QUIC=NO -D USE_QUICHE=NO -D USE_WIN32_IDN=YES -D USE_WIN32_LARGE_FILES=YES -D USE_WIN32_LDAP=NO -D ZLIB_LIBRARY=T:/Library/zlib-1.3.1/usr/lib/Windows/x86_64/zlibstatic.lib -D ZLIB_ROOT=T:/Library/zlib-1.3.1/usr
-- Using CMake version 3.29.2
-- The C compiler identification is MSVC 19.39.33523.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.39.33519/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- curl version=[8.9.1-DEV]

[toffaletti]

And I see it installed curlver.h:

[169/171][ 98%][2.949s] Building C o-- Install configuration: "Release"
-- Installing: T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/libcurl.lib
-- Installing: T:/Library/curl-8.9.1/usr/bin/mk-ca-bundle.pl
-- Installing: T:/Library/curl-8.9.1/usr/bin/curl-config
-- Installing: T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/pkgconfig/libcurl.pc
-- Installing: T:/Library/curl-8.9.1/usr/include/curl
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/curl.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/curlver.h <---------------
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/easy.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/header.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/mprintf.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/multi.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/options.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/stdcheaders.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/system.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/typecheck-gcc.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/urlapi.h
-- Installing: T:/Library/curl-8.9.1/usr/include/curl/websockets.h
-- Installing: T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL/CURLTargets.cmake
-- Installing: T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL/CURLTargets-release.cmake
-- Installing: T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL/CURLConfigVersion.cmake
-- Installing: T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL/CURLConfig.cmake
bject lib\CMakeFiles\libcurl_object.dir\vtls\x509asn1.c.obj
[170/171][ 99%][3.026s] Linking C static library lib\libcurl.lib
[170/171][ 99%][3.026s] Install the project...
[2025-02-03 19:37:23] Finished building 'C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\curl' to 'T:\x64\Windows\curl-8.9.1' for arch 'x86_64' in 00:00:11.8171568

[toffaletti]

[2025-02-03 19:47:27] Building 'C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\swift-corelibs-foundation' to 'T:\103' for arch 'x86_64'...
cmake.exe -B T:\103 -S C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\swift-corelibs-foundation -G Ninja -D _SwiftCollections_SourceDIR=C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-collections -D _SwiftFoundation_SourceDIR=C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-foundation -D _SwiftFoundationICU_SourceDIR=C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-foundation-icu -D CMAKE_ASM_COMPILE_OPTIONS_MSVC_RUNTIME_LIBRARY_MultiThreadedDLL=/MD -D CMAKE_ASM_COMPILER=T:/5/bin/clang-cl.exe -D CMAKE_ASM_FLAGS=--target=x86_64-unknown-windows-msvc -D CMAKE_BUILD_TYPE=Release -D CMAKE_C_COMPILER=T:/5/bin/clang-cl.exe -D CMAKE_C_COMPILER_TARGET=x86_64-unknown-windows-msvc -D CMAKE_C_FLAGS=/GS- /Gw /Gy /Oi /Oy /Zc:inline -D CMAKE_CXX_COMPILER=T:/5/bin/clang-cl.exe -D CMAKE_CXX_COMPILER_TARGET=x86_64-unknown-windows-msvc -D CMAKE_CXX_FLAGS=/GS- /Gw /Gy /Oi /Oy /Zc:inline /Zc:__cplusplus -D CMAKE_FIND_PACKAGE_PREFER_CONFIG=YES -D CMAKE_INSTALL_PREFIX=T:/x64/Windows.platform/Developer/SDKs/Windows.sdk/usr -D CMAKE_NINJA_FORCE_RESPONSE_FILE=YES -D CMAKE_Swift_COMPILER=T:/5/bin/swiftc.exe -D CMAKE_Swift_COMPILER_TARGET=x86_64-unknown-windows-msvc -D CMAKE_Swift_FLAGS=-vfsoverlay T:/101/stdlib/windows-vfs-overlay.yaml -strict-implicit-module-context -Xcc -Xclang -Xcc -fbuiltin-headers-in-system-modules -resource-dir T:/101/lib/swift -L T:/101/lib/swift/windows -gnone -Xlinker /INCREMENTAL:NO -Xlinker /OPT:REF -Xlinker /OPT:ICF -D CMAKE_Swift_FLAGS_RELEASE=-O -D CMAKE_Swift_FLAGS_RELWITHDEBINFO=-O -D CURL_DIR=T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL -D dispatch_DIR=T:/102/cmake/modules -D ENABLE_TESTING=NO -D FOUNDATION_BUILD_TOOLS=YES -D LibXml2_DIR=T:/Library/libxml2-2.11.5/usr/lib/Windows/x86_64/cmake/libxml2-2.11.5 -D SwiftFoundation_MACRO=T:/2/bin -D SwiftSyntax_DIR=T:/5/cmake/modules -D ZLIB_INCLUDE_DIR=T:/Library/zlib-1.3.1/usr/include -D ZLIB_LIBRARY=T:/Library/zlib-1.3.1/usr/lib/Windows/x86_64/zlibstatic.lib
-- The C compiler identification is Clang 19.1.5 with MSVC-like command-line
-- The Swift compiler identification is Apple 6.2
<unknown>:0: warning: using (deprecated) legacy driver, Swift installation does not contain swift-driver at: 'C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\build\5\bin\swift-driver-new.exe'
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: T:/5/bin/clang-cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working Swift compiler: T:/5/bin/swiftc.exe
-- Check for working Swift compiler: T:/5/bin/swiftc.exe - works
-- The CXX compiler identification is Clang 19.1.5 with MSVC-like command-line
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: T:/5/bin/clang-cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- _SwiftFoundationICU_SourceDIR provided, using swift-foundation-icu checkout at C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-foundation-icu
-- _SwiftCollections_SourceDIR provided, using swift-foundation-icu checkout at C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-collections
-- SwiftFoundation_MACRO provided, using macros in T:/2/bin
<unknown>:0: warning: using (deprecated) legacy driver, Swift installation does not contain swift-driver at: 'C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\build\5\bin\swift-driver-new.exe'
-- Performing Test LINKER_SUPPORTS_BUILD_ID
-- Performing Test LINKER_SUPPORTS_BUILD_ID - Success
-- Looking for strlcat
-- Looking for strlcat - not found
-- Looking for strlcpy
-- Looking for strlcpy - not found
-- Looking for issetugid
-- Looking for issetugid - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
-- Found ZLIB: T:/Library/zlib-1.3.1/usr/lib/Windows/x86_64/zlibstatic.lib (found suitable version "1.3.1", minimum required is "1")
-- Configuring done (10.7s)
-- Generating done (0.5s)
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_ASM_COMPILER
    CMAKE_ASM_COMPILE_OPTIONS_MSVC_RUNTIME_LIBRARY_MultiThreadedDLL
    CMAKE_ASM_FLAGS
    CMAKE_NINJA_FORCE_RESPONSE_FILE
    ENABLE_TESTING
    SwiftSyntax_DIR


-- Build files have been written to: T:/103

cmake.exe -B T:\103 -S C:\Users\swift-ci\jenkins\workspace\swift-corelibs-foundation-PR-windows\swift-corelibs-foundation -G Ninja -D _SwiftCollections_SourceDIR=C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-collections -D _SwiftFoundation_SourceDIR=C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-foundation -D _SwiftFoundationICU_SourceDIR=C:/Users/swift-ci/jenkins/workspace/swift-corelibs-foundation-PR-windows/swift-foundation-icu -D CMAKE_ASM_COMPILE_OPTIONS_MSVC_RUNTIME_LIBRARY_MultiThreadedDLL=/MD -D CMAKE_ASM_COMPILER=T:/5/bin/clang-cl.exe -D CMAKE_ASM_FLAGS=--target=x86_64-unknown-windows-msvc -D CMAKE_BUILD_TYPE=Release -D CMAKE_C_COMPILER=T:/5/bin/clang-cl.exe -D CMAKE_C_COMPILER_TARGET=x86_64-unknown-windows-msvc -D CMAKE_C_FLAGS=/GS- /Gw /Gy /Oi /Oy /Zc:inline -D CMAKE_CXX_COMPILER=T:/5/bin/clang-cl.exe -D CMAKE_CXX_COMPILER_TARGET=x86_64-unknown-windows-msvc -D CMAKE_CXX_FLAGS=/GS- /Gw /Gy /Oi /Oy /Zc:inline /Zc:__cplusplus -D CMAKE_FIND_PACKAGE_PREFER_CONFIG=YES -D CMAKE_INSTALL_PREFIX=T:/x64/Windows.platform/Developer/SDKs/Windows.sdk/usr -D CMAKE_NINJA_FORCE_RESPONSE_FILE=YES -D CMAKE_Swift_COMPILER=T:/5/bin/swiftc.exe -D CMAKE_Swift_COMPILER_TARGET=x86_64-unknown-windows-msvc -D CMAKE_Swift_FLAGS=-vfsoverlay T:/101/stdlib/windows-vfs-overlay.yaml -strict-implicit-module-context -Xcc -Xclang -Xcc -fbuiltin-headers-in-system-modules -resource-dir T:/101/lib/swift -L T:/101/lib/swift/windows -gnone -Xlinker /INCREMENTAL:NO -Xlinker /OPT:REF -Xlinker /OPT:ICF -D CMAKE_Swift_FLAGS_RELEASE=-O -D CMAKE_Swift_FLAGS_RELWITHDEBINFO=-O -D CURL_DIR=T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL -D dispatch_DIR=T:/102/cmake/modules -D ENABLE_TESTING=NO -D FOUNDATION_BUILD_TOOLS=YES -D LibXml2_DIR=T:/Library/libxml2-2.11.5/usr/lib/Windows/x86_64/cmake/libxml2-2.11.5 -D SwiftFoundation_MACRO=T:/2/bin -D SwiftSyntax_DIR=T:/5/cmake/modules -D ZLIB_INCLUDE_DIR=T:/Library/zlib-1.3.1/usr/include -D ZLIB_LIBRARY=T:/Library/zlib-1.3.1/usr/lib/Windows/x86_64/zlibstatic.lib

-D CURL_DIR=T:/Library/curl-8.9.1/usr/lib/Windows/x86_64/cmake/CURL

[toffaletti]

OK, my theory on why this is happening is that the CMake build being used on Windows for Foundation only includes libcurl for the _CFURLSessionInterface target:

target_link_libraries(_CFURLSessionInterface PRIVATE
    dispatch
    CURL::libcurl)

However, because this is marked PRIVATE the curl include directory isn't added when building FoundationNetworking which depends on _CFURLSessionInterface:

target_link_libraries(FoundationNetworking
    PRIVATE
        _CFURLSessionInterface
    PUBLIC
        Foundation)

I suspect it is done this way because Foundation is going to great pains to prevent any leakage of libcurl into the ABI. I noticed this and made sure to use curlver.h (which only has cpp #define in it) instead of curl.h. At this point I'm not sure what the solution should be.

Options:

1. guard the include of curlver.h to exclude Windows, leaving https broken on Windows.
2. add libcurl to FoundationNetworking as a PRIVATE library dependency
3. break compatibility with libcurl version prior to 7.84 (2.5 years old now)
4. move the definition of NS_CURL_CURLINFO_CAINFO_SUPPORTED into Sources/_CFURLSessionInterface/CFURLSessionInterface.c and make Sources/FoundationNetworking/URLSession/libcurl/EasyHandle.swift use the value of CFURLSessionInfoCAINFO inside curlInfoCAInfoSupported

#if NS_CURL_CURLINFO_CAINFO_SUPPORTED
CFURLSessionInfo const CFURLSessionInfoCAINFO = { CURLINFO_CAINFO };
#else
CFURLSessionInfo const CFURLSessionInfoCAINFO = { CURLINFO_NONE };
#endif

[parkera]

@compnerd @jmschonfeld any advice here?

[compnerd]

Moving the curl dependency to FoundationNetworking would require introducing further @_implementationOnly imports as private imports are effectively useless (they require the module to be present on disk so we would need to ship the C headers at the very least).

I don't think that breaking compatibility with older versions of CURL is a concern here. We are building against specific versions of curl for every distribution and having a minimum required version is very reasonable.

I think that the only reasonable options from the four suggested are 3 and 4: either break compatibility with older CURL or sink the needed information into _CFURLSessionInterface.

[jmschonfeld]

I agree with what Saleem mentioned - option 2 requires adding a new @_implementationOnly import which we've seen is problematic in non-resilient modules so I'm hesitant to play with fire there. How difficult/feasible would it be to go with option 4 (using the required symbols in the _CFURLSessionInterface shim)? Does it actually require types to be exposed up to FoundationNetworking or is it something we could add a shim for without FoundationNetworking being aware of the curl headers themselves?

[toffaletti]

I'll start a new branch with a hybrid option 3 & 4 approach to see how it goes.

[toffaletti]

@compnerd @jmschonfeld how's this?

https://github.com/swiftlang/swift-corelibs-foundation/compare/main...toffaletti:swift-corelibs-foundation:fix-curl-version-check-option4?expand=1