Now you can get everything on Safari. To purchase books, visit Amazon or your favorite retailer. If you need help writing programs in Python 3, or want to update older Python 2 code, this book is just the ticket. Packed with practical recipes written and tested parsing patch data python Python 3.
3, this unique cookbook is for experienced Python programmers who want to focus on modern tools and idioms. All trademarks and registered trademarks appearing on oreilly. Due to the lack of introspection in the language, implementing the likes of reflection, script binding and serialization requires writing some sort of boilerplate that essentially keeps the data which is otherwise thrown away by the compiler. What I’m presenting here is not meant to be taken as the concrete implementation I made but rather as one possible way of solving a problem, a detailed example. Using these ideas, it is possible to create an all-encompassing reflection solution or to generate code for existing libraries of any purpose. I can’t go on without mentioning Eli Bendersky’s excellent post on the topic, which served as a great resource when I started gathering information.
The example problem In our example, we are implementing automatic script binding, so we don’t need to write and maintain binding code by hand. We also want to be able to omit certain parts of the source so that they are not taken into account when the binding boilerplate code is generated2. Now let’s see the simple solution. Python, a seasoned and battle-tried library, which allows us to write binding code in a very expressive manner. The following is the entire binding code in a separate source file which we will link with our executable.
The above code expresses exactly what we wanted to achieve: one constructor, and two member functions. All of the above is what we would do in a typical project to make a class scriptable. Our aim is to generate this code automatically. Traversing the AST is performed with cursors. The first cursor you need points to the root of the translation unit, that is, the file you are parsing. The parameters of the parse call The parse function takes a filename and a list of compiler flags.
Comparing this syntax tree dump to the source above should help with understanding what libclang does when parsing the source code. The libclang C API has a visitor-based way of traversing the AST. This is also available from the Python bindings via the function clang. Cursor_visit, but we are going to utilize the more Pythonic, iterator-based approach, which is an addition of the Python bindings. T’ stands for tree in the abbreviation, and the most straightforward way to process such a data structure is to recursively traverse it. Useful properties of cursors Cursors provide a rich set of information about each node in the AST.
The kind of node we are looking at in the AST. Answers questions like: Is this a class declaration? The literal text defining the token. Similar to spelling, but the displayname also contains some extra information which helps to distinguish between identically spelled tokens, such as function overloads. The source location where the node was found. This can be used to filter out included contents from the source file being parsed, because usually we are interested in that. If you are implementing something different, you might find the following properties useful, too: location,extent.
Sometimes the only way to get a particular string is to read the source directly. With location and extent you can find the exact point in the file that you need to read. It all really boils down to traversing a tree and filtering for certain elements. Function object using the information found in that node. This is a very simplistic code model: classes have names and member functions, and member functions have names. It is possible to gather much more than that, but for our purposes, this is mostly enough.
By the way, that above is almost half of all the code we need in our code generator! Now let’s see the other half: how the classes are built. One important step here is that we are checking the location of the node. This ensures that we are only taking the contents of the file being parsed into account.
Otherwise, we would end up with a huge mess of an AST due to the includes being , well, included. Code generation Now that we have a code model, we can easily process it and generate the code we need. This approach can work and is easy to implement, albeit not very elegant. We are going to do something way cooler: we will use templates to generate our code. Of course you already saw we are using templates in the Boost. What is so cool about that?