support WINDOWS_EXPORT_ALL_SYMBOLS like cmake #2132

Comments

santagada commented Jul 28, 2017

That uses a coff symbol dumper (cmake internal command) that creates .def files for .obj files exporting every symbol in them. That is the only simple way to generate complete shared libraries (.dll on windows) with MSC as it only links exported symbols (explicit exports or using this .def file).

Does meson already supports this? If not this is a feature request, I can help with providing a C++ tool to do the exports either by writting it myself or just repackaging the CMake code.

The text was updated successfully, but these errors were encountered:

guruDanny67 commented Aug 2, 2017

gtkmm does this (also on the companion glibmm, pangomm, cairomm, . ) using a program named gendef, compiled from gendef.cpp in any of the projects. This one uses dumpbin.exe from the Visual Studio installation.

nirbheek commented Aug 4, 2017

This is generally very bad practice to do, since it will export all symbols, which means you cannot enforce ABI/API stability. I don’t think we should support this, especially since you can always keep a static definitions list if you really don’t want to use __declspec(dllexport) .

santagada commented Aug 5, 2017

The use we have is to create shared libraries to internal applications that take many minutes to link. So we don’t care about defining a ABI/API. IIRC that behavior is supported using GCC, its just MSC not even allowing you to use it.

guruDanny67 commented Sep 8, 2017

Maybe is enought to have a prelink and a postlink command so this function can be implemented by the user and can be useful also in another situations (for example if someone wants to digitally sign the executable / dll at the end of the link phase).

rib commented Dec 27, 2018

Just to note that not having an equivalent feature can make it more tricky to build existing third-party libraries that use cmake and use WINDOWS_EXPORT_ALL_SYMBOLS, such as libflann.

Even though I agree it seems like poor practice to use this option; if it’s a third party library which I don’t maintain then I don’t really want to be patching libflann to improve how it handles exports.

I think the reason libflann uses it is because they themselves are embedding a third-party lz4 decompression API in their library and expect it to be exported since their public headers then depend on the LZ4_ symbols. The lz4 implementation they embed doesn’t have any kind of symbol export defines.

Seems like a very questionable design choice and I don’t want to defend it — just adding a data point for considering whether there’s some value in a feature like this, even if only for the sake of simplifying building/using existing projects currently using cmake, with minimal changes to an upstream project.

How to tell the MinGW linker not to export all symbols?

I’m building a Windows dynamic library using the MinGW toolchain.

To build this library I’m statically linking to other 2 which offer an API and I have a .def file where I wrote the only symbol I want to be exported in my library.

The problem is that GCC is exporting all of the symbols including the ones from the libraries I’m linking to. Is there anyway to tell the linker just to export the symbols in the def file?

I know there is the option —export-all-symbols but there seems not to be the opposite to it.

Right now the last line of the build script has this structure:

EDIT: In the docs about the linker it says that —export-all-symbols is the default behavior and that it’s disabled when you don’t use that option explicitly if you provide a def file, except when it doesn’t; the symbols in 3rd party libs are being exported anyway.

EDIT: Adding the option —exclude-libs LIBS or –exclude-symbols SYMBOLS doesn’t prevent the symbols from the libraries from being exported.

6 Answers 6

You can use dllwrap if your distribution of binutils (either native or cross compiling) provides it.

It can produce DLLs using the interface in a DEF file (under the hood it calls gcc, ld and dlltool to do so). The difference between using this and passing a DEF file to GCC directly is that the definitions in the file are treated differently.

For example, if you have a symbol rename in the export file:

GCC will create 2 exports, one by the name of _SomeFunction and the other one with the decorated name while dllwrap will only export _SomeFuntion . So if you only add to the DEF file the symbols you want to be exported you will end up only with them in the library.

dllwrap by default uses the C compiler driver since it has no way of knowing otherwise. As you’re linking C++ code, you have to use the option —driver-name c++ to set the driver. If you happen to have the MinGW executables with a prefix you have to include it too in the driver name (e.g. i686-mingw32-c++ instead of c++ ) and you may need to use the option —dlltool-name too.

Try using these two lines instead of the one you posted:

The first one generates an object file from the code of library.cpp and the second one assembles the dynamic library. The OBJECT_FILES thing (which I assume to be other object files you generated previously) should have library.o there too.

That said, I have to tell you dllwrap was already deprecated in 2006 and there is no documentation on it in the official binutils package; to get some info you may call it with —help as usual. It can generate an import library in case you need it too.

Create dlls on Windows without declspec() using new CMake export all feature

CMake 3.4 will have a new feature to simplify porting C and C++ software using shared libraries from Linux/UNIX to Windows. Linux/UNIX developers are often surprised to learn that creating a shared library on Windows known as a DLL (dynamic linked library) requires changes to the source code or an explicit listing of all the symbols that the dll will export. The compilers on Linux/UNIX have the ability to export all symbols in a shared library automatically. On Windows, you must either use compiler directives __declspec(import) and __declspec(export) to declare which symbols are exported/imported from a shared library, or you must create a module definition text file (.def) with a list of all the symbols you want to export and pass that file to the linker.

With C libraries, creating a .def file by hand or automatically is not very difficult since you just have to list the names of all the functions in the library. However, with C++ code, name mangling and the sheer number of functions make crafting a .def file by hand nearly impossible. The standard workaround uses the preprocessor to conditionally insert __declspec(import) and __declspec(export) into the code. However, with a large existing C++ code base, it can be difficult and time consuming to edit all of the source code. CMake now has a feature which allows it to query .obj files that will make up a DLL and create a .def file automatically, in most cases without needing to modify the original source code.

The feature is implemented in CMake via a new target property, WINDOWS_EXPORT_ALL_SYMBOLS. When enabled, this property causes CMake to automatically create a .def file with all symbols found in the input .obj files for a SHARED library on Windows. The .def file will be passed to the linker causing all symbols to be exported from the DLL. For global data symbols, __declspec(dllimport) must still be used when compiling against the code in the DLL. The symbol is exported from the DLL correctly and automatically, but the compiler needs to know that it is being imported from a DLL at compile time. All other function symbols will be automatically exported and imported by callers. This simplifies porting projects to Windows by reducing the need for explicit dllexport markup, even in C++ classes. This property is initialized by the value of the CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS variable when a target is created.

To try this out on an existing project, run

In GCC, how can I export all typeinfo symbols for a shared library without exporting all symbols?

Here is the problem:

I have a shared library that is hiding symbols by default. Actually, it uses the -Xlinker —version-script= option to export some symbols in a specific file but hide all the rest. The issue is that if we try to catch exceptions that were defined in other shared libraries, we get problems because the typeinfo is hidden by this shared library. I can’t explicitly make the exception classes visible because they are defined in an open source library that is out of my control. I also don’t want to explicitly list the mangled names for the typeinfo of the exception classes in the version-script symbol file because there are many developers working on this library and they are bound to forget to do this for some exception class in the future.

I’ve tried using the #pragma GCC visibility push(default) before the open source .hpp files but it didn’t seem to work — the symbols were not visible.

I’ve also tried using -fvisibility-ms-compat which did export the typeinfo symbols but also exported tons of other symbols that I did not want exported. The library has both C and C++ code in it so I’m not sure how it is supposed to work with this option that is only for C++.

What I really need is an option that will make all typeinfo symbols visible, while leaving other symbols hidden (unless they are explicitly visible). Is there an option like this?

5. EXPORT_SYMBOL¶

5.1. Introduction¶

Generally the modules will never live alone. We need to divide the code into multiple modules for better organization and readability as well as we need to use the APIs or functionality which is available in other modules or the functions which are made available to us by the Linux Kernel.

Here in this chapter we will see how to make our functions available to other modules. In latter chapters we will use some functionality given by the other modules or the kernel code.

5.1.1. Why this chapter¶

We need to understand how to

• For making a function available for others to use.
• How to use functions given by other modules.

5.1.2. What will you learn¶

• Exporting an API to an external module.
• Using API given by the other module.
• Using library functions available in kernel.

5.1.3. Prerequisites¶

5.2. Export Symbol¶

• EXPORT_SYMBOL() helps you provide APIs to other modules/code.
• The functions which you EXPORT are available to the other modules/code.
• Your module will not load if the its expecting a symbol(variable/function) and its not present in the kernel.
• modprobe helps here and loads the modules which is needed by your module.
• What if there is circular dependency between the modules?

5.3. Module Exporting Some Functions and Variables¶

5.3.1. Introduction¶

• Here we will write two modules. In one module we will have the functions which will be exported using the EXPORT_SYMBOL() whereas the other module will just call the functions and use the variables which are exported.
• We will then see the details of the module by seeing the modinfo command . See the depends field of the output. In mymodule1.ko you will see that it depends on mymodule1 .

5.3.2. FILE: mymodule1.c ¶

5.3.3. FILE: mymodule2.c ¶

5.3.4. FILE: Makefile ¶

• Let us see what modinfo tells about our modules. Before this compile the modules.

• Let us try to insert the mymodule2.ko before the mymodule1.ko . It will give errors.

• SUCESSS !! You have successfully inserted a module which uses functions from another module.

5.4. Removing the modules¶

• You cannot remove a module which is in use.

• Check who is using the mymodule1 . See the Used by column in the lsmod output.

• We will have to remove the mymodule2 first and mymodule1 .

5.5. Other files¶

• See the Module.order file. It has the order in which the modules should be loaded.

• See the Module.symvers file. It shows the symbols which are exported.

5.6. See the exported symbols¶

Module1 exports the symbols.

The exported symbols and other functions in the kernel can be seen in the /proc/kallsyms file. Its is a huge file.

Let us see the difference in the file after inserting the mymodule1.ko .

That file clearly that the functions print_hello() and others are from mymodule1 .

The UpperCase T says that the functions are exported (available for others to use) while a lowercase says its not exported .

Run the following commands to make two files with the list of symbols.

cat /proc/kallsyms > /tmp/1

Insert our module.

Save the symbols in another file.

cat /proc/kallsyms > /tmp/2

See the difference.

5.7. Tool modprobe¶

• modprobe understands in which order the modules are to be loaded.
• First remove the modules.
• Run the command modprobe module2 loads both the module.

5.8. Tool — depmod¶

• DONT RUN IT depmod is smart enough to find the dependencies and write to a file — don’t run it as it will overwrite the original file. First take backup of the file. depmod ABSOLUTE_PATH_OF_THE_MODULE1 ABSOLUTE_PATH_OF_THE_MODULE2 see the file /modules/3.2.0-23-generic/modules.dep

5.9. One module dependent on several modules¶

5.9.1. modprobe automatically loading all the modules.¶

5.10. String related functions available in kernel¶

• There are a lot of function related to string operations available in the Linux Kernel for you to use. They all are exported.
• See the output of /proc/kallsyms .

• The count of exported functions is

• Let us now see how can we make use of one of the string functions that is strcat() . In the following module we will just concatenate two strings and will print the output.

5.10.1. FILE: mystring.c ¶

5.10.2. FILE: Makefile ¶

5.11. Exercises¶

The exercises here will generally not make much sense with respect to kernel development. You will not be writing a feature for the kernel but you will be learning how to do the basics. So you MUST do it.

1. Write a kernel module to which we can pass a string and it does the following. It must have the functions exported so that another kernel module can use the functions. #. Find the length of the string. mystring_find_length() #. Returns the reverse of the string. mystring_get_reverse() #. Returns the rotation of the string by 3 places. char *mystring_get_rotated(char *srcstr, char *deststr, int rotations, int direction) #. Returns if the string is palindrome or not. int mystring_is_palindrome(char *str) #. Returns a character array where you have saved only the characters which are present in the even indexes. #. Returns a string which has all the letter capitalized. #. Returns a string which has all the letter converted to lowercase.
2. For the above kernel module write a testcase module which will call the functions and test if the functions are working correctly.

© Copyright 2016, Rishi Agrawal. Revision d9ac4e81 .