Advanced Python plugin coding for Cinema 4D

It’s been a long time… how have you been?

This is going to be another Python/Cinema 4D related post.

While python is great for quickly scripting workflow optimizing tools, prototype stuff or write generators and tags, is it suited for big, complex plugin development?

It is!

You get ‘nearly’ full access to the C++ API of Cinema 4D, don’t have to compile or worry much about platform dependencies, can easily find a ton of neat recipes and libraries out there…

So it’s not only possible to code full-fledged python plugins but it even has its advantages over classical C++ development.

Imagine you wrote a nifty script, decided to make it a simple plugin, and now want to take it a step further – add more features, a complex GUI with some dialogs etc…

If you are still working with the script editor by now, things will most likely start to get out of hand as you introduce more modules and classes, deal with more code and more functionality.

This is the point where you might want to look out for a little help.

We are going to set up an IDE (Integrated Development Environment), talk a bit about modules/packages and finally I will try to give some general hints and tips.

Coding Environment/IDE:

Prepare Cinema 4D:

You may skip this part, but I highly recommend it:

  • Make a copy of your Cinema 4D installation – name it “Cinema 4D R12 Dev” for example.
  • Get rid of all Cinema 4D plugins you don’t need for your development
  • Place an empty text file named “c4d_debug.txt” into the root folder of your Cinema 4D copy.

This will slow Cinema 4D down a bit, but enables you to monitor memory leaks.

We will add some useful things on Cinema 4D-side later – for now let’s continue with the Eclipse setup…

Python:

In order to get auto completion to work within PyDev, we are going to need a separate python interpreter – 2.6.4 to be exact.
Just download an install it from here.

Eclipse:

If you never used one, think of an IDE as your workshop, it’s a collection of tools as well as the place where you do and organize your coding work.

Eclipse is one of the most popular – its Java based, open source, free to use and most importantly has excellent support for python.

So let’s start by downloading and installing Eclipse Classic from http://www.eclipse.org/downloads/
If you don’t have the Java Runtime Environment installed jet, you need to do this first.

PyDev:

As said before, Eclipse has excellent python support – but not out of the box, so we are going to install PyDev.

PyDev will help you coding by adding syntax highlighting, code completion and much more.
Should you encounter any trouble with the following steps, please refer to the PyDev installation manual.

  • After installation, startup Eclipse, click “Help”, and chose “Install New Software…”.
  • Next click the “Add…” button, type in “PyDev” as name and “http://pydev.org/updates” as location – then confirm.
  • Now enter “PyDev” in the “Work with:” field, you should see an entry “PyDev for Eclipse” – check it and click “Next”.

After confirming again and accepting the terms, it will download and install.

  • Click “Windows” and open the “Preferences” dialog.
  • Select PyDev and the child “Interpreter – Python”
  • Click “New…”, input “Python Cinema 4D” as name and browse to your interpreter installation.
  • Select the python.exe.
  • Confirm your selection and the paths PyDev wants to add to your syspath.

So much for the PyDev installation itself, now we have to associate the Cinema 4D Python plugin extension *.pyp with PyDev to enable syntax highlighting and code completion for those files as well:

  • Open ‘Window’ within the file-menu and click ‘Preferences’
  • Select ‘General’ -> ‘Content Types’ from the list on the left.
  • In the ‘Content types:’ view on the right, unfold ‘Text’ and click ‘Python File’
  • Hit the ‘Add…’ button of the ‘File associations:’ list on the lower right, type “*.pyp” into the popup and confirm.
  • Open ‘Window’ and click ‘Preferences’
  • Select ‘PyDev’ -> ‘Editor’ -> ‘Code Style’ -> ‘File Types’ from the list on the left.
  • In the field ‘Valid source files (comma-separated):’ on the right, add , pyp and confirm.

Its helpful to add a fake version of the Cinema 4D Python API’s “c4d” package to the ‘/Lib/site-packages/’ folder of the interpreter to enable auto-completion – grab it here.

These are fake modules and they have their drawbacks – read more about them in this blog post.

Project setup

We take your IDE for a spin and create the first project:

  • Click “File” – “New” and chose “PyDev Project”
  • Input a Name for your Project and uncheck the “Use default” box below.
  • Browse to your Cinema 4D installations plugin folder, create a new directory with your plugins name and select it.
  • Select “2.6” from the “Grammar Version”- dropdown and leave the Interpreter set to default.
  • Last but not least uncheck the “Create default ‘src’ folder…” checkbox and hit “Finish”.

To setup the needed folder structure, header- resource- and string files for your plugin, please read the Cinema 4D Python documentation “Plugin structure” –  especially “Directory structure” – and take a look at the Python plugin examples within the documentation – or the experimental plugin from this post:

Just remember this part is vital, so do not rush through it – at least read the section mentioned above and take a good look at the SDK examples.

Only if you have built C++ plugins before and are already familiar with the structure, you may…

Hint:

You can also add your existing project files by just placing them within the plugin-directory we just created, or dragging them into the project-folder within Eclipse.

When organizing the file structure within your project, you are working with your file system – changes you do here are reflected on your harddisk.
This also works the other way round, move something on the file system and Eclipse will update those changes to your project – while not always instantly.

You can press F5 within Eclipse to update a folder and its subfolders or the contents of a file.

Don’t worry, if the project-view is out of sync with the file system – Eclipse will tell you.

A coders Cinema 4D:

We are all set with the IDE part – let’s tune the Cinema 4D installation a bit more to improve the workflow and finalize your coding environment.

If you didn’t add/import your existing code to the project we just created, write a simple hello world and fire up Cinema.

It’s a good idea to setup a developer friendly layout, I like adding the console window to the tabs on the right, next to the object manager – or even better to a side monitor.
Should you be writing a CommandData Plugin, consider opening the Command Manager and dragging its icon into the layout, so you can quickly re-open it.

You will be launching and closing Cinema 4D a lot during development – it’s helpful to have a strategy to quickly start and close it.
Place a link to Cinema 4D on your taskbar/dock, close it using ALT+F4/Command+Q or create a little python script containing “exit()” and add it to the layout/create a shortcut.

Finally, open your Plugin as well as everything you need to test its functionality and save the layout as startup layout.

If you have a test document that you need to check your plugin, press CTRL+E, click the “Open Preferences Folder…” button and place it as “default.c4d” within that folder so it will load on startup.

You long got the idea – try to prepare everything so you don’t have to spend time doing recurring tasks on every unit-test you are going to make during development.

Some additional tools:

It’s often helpful to keep an eye on memory consumption and or active threads so I always have the task manager/activity monitor in reach when coding.

And while not really necessary, it’s convenient to have a good, fast text editor with Python syntax highlighting so you are able to read a file or snippet without having to launch the IDE first.
I can recommend Notepad++ and TextMate or Emacs for this job.

Working with eclipse:

Eclipse in combination with PyDev is a mighty! tool – Aside from the obvious killer features – syntax highlighting and code completion, there is a ton of others.

Often you will discover some that will make you feel bad for not having used them right from the start, so I’ll try to point out the most useful, while I encourage you to read up on Eclipse and PyDev.

The Find/Replace Dialog

Press CTRL+F for all your search and replace needs – just listing it to make you aware of the direction-, selection- and regex support.
If you haven’t used regular expressions jet, you should directly head over to Wikipedia – you won’t regret it.

Even more powerful is the…

Search Dialog

CTRL+H opens the global or project wide search – here you can also replace over all files.

Just keep in mind that the “PyDev Search” tab does only look within *.py files while the “File Search” tab by default searches over a *.* pattern.

Compare feature and history

Just select two files, right click and chose “Compare with” -> “Each other”.

You can also compare a file with an earlier state of itself by choosing “Local history”.

This is another huge plus of using an Eclipse – don’t worry anymore about changing huge sections of code or commenting it – just go ahead and delete it – you can always return to an earlier state using the “History”- tab.

But use this with care, as it’s possible to lose the history by moving or re-creating a project and it’s not infinite.

Tabbed views

You can drag a tab to one side of the code view until a small arrow appears – the tab will then snap to that side, splitting your view so you can work with multiple source files at the same time.

This is very useful when working with the header-, resource- or string files of your project as you can code on the left and copy ‘n paste ID’s from the right for example.

Coding python for Cinema 4D:

We have setup the coding environment – how about doing some programming now?

This will be a loose collection of tips that are all more or less Cinema 4D related, but mostly  good practice in general.

Modules and namespace:

First – this is OOP (Object Oriented Programming) and your project’s structure should reflect that.

In general, classes and modules are a good thing, it’s a far more common fault to have less abstraction than too much.

Packages/Modules are also the key to keep your code organized, manageable – AND reusable.

So if you are going to write something that you might want to use again later, consider spending the extra time to make it a bit more user/coder-friendly and build a class or module.

When working with python and modules you have to keep a close eye on naming and namespace, especially in context with the Cinema 4D SDK.

Be creative and use unique module names – this will keep things easier when you are using other modules – and you will – at least the ones from the Cinema 4D API.
Also be careful when importing from modules – it’s rarely a good idea to just import everything (from foo import *) – instead carefully select the members you really need and give everyone its own import line rather than stacking them with commas in one.

It’s often better to just import the module into your namespace and reference the members via module.foo – you don’t pollute your namespace and it’s more readable.

In any case you should avoid the API’s module AND member names like the plaque.

For example if you extend the GeDialog Class and have a method “def close(self):” – but make the slight mistake to call it as “Close()”, you might end up with a similar behavior but wonder for minutes while your dialog just closes without saving it’s settings first (or whatever “close()“ should have done).

I know for some this is basic stuff – many have their own patterns for this and that’s ok!

Whatever you do – be consistent and aware of your namespace.

Symbols:

As you need to keep track of the enums within the c4d_symbols.h, it is best to define every integer by yourself and not auto-enumerate them.

Then mirror those symbols to a *.py file called “ids.py” and be careful to update both should you change them or introduce new ones.

Instead of hardcoding ID’s, you should always use the symbol within the c4d-package.
Should you ever need to identify a symbol by it’s integer value (for example if you want to check for a certain flag that is returned) – this little script will help you out:

import c4d;
from c4d import gui; 

def main():
    wanted = gui.RenameDialog("ID to search for...")
    outp = ""

    for itm in dir(c4d):
        s = getattr(c4d, itm)
        if s == int(wanted):
            outp += "%s = %s\n" % (itm, s)

    gui.MessageDialog("Sorry - none found..." if outp == "" else outp)

if __name__=='__main__':
    main()

API calls / performance:

Don’t get me wrong – the Python API is blazing fast – so fast you often can’t tell the difference to C++ – but still, API calls take time.
When working with thousands of objects, this quickly adds up no matter how fast each individual call is.

Try to limit those calls in sections of your code, where you really need performance – look for ways to keep states within your own plugin logic rather than querying them form Cinema 4D whenever possible.

If you are looking for bottlenecks, be sure to check out cProfile – it’s great.

Prototyping:

Using an IDE doesn’t forbid you to use Cinema 4D’s script editor and console to quickly whip up and test something – you can even write and directly execute a simple test script within eclipse if you are working on parts of your plugin that don’t need the Cinema SDK (some kind of parser for example) and take advantage of the PyDev debugger.

Debugging:

The lack of a real debugger can be compensated by using the PyDev remote debugger, pythons inspect module and the logging package.

Remember that there is a limit to the amount of characters the Cinema 4D console can output for a single print command.

Also take a look at sys.exc_info()

Garbage Collection:

When introducing circular references you should use weak references to give the garbage collector a hand.

Compilation:

How about shipping your plugin as precompiled byte code?

This will make Cinema 4D start up faster – as the compilation of your plugin can be skipped, obfuse your source and still be platform interoperable.

Just copy your plugin, open the console/terminal and execute “python.exe –mcompileall C:\PathToMyPlugin” and delete all *.py files afterwards.

Note that this might not work in some cases, so be sure to test your plugins functionality again.

Platform dependency:

While Python is platform-independent, there are still some things to keep in mind.

Don’t use backslashes in paths but slashes – they are understood by windows AND mac OS.

You can also use platform.system() to determine on what OS your plugin runs on and import modules after a conditional check – for example:

if platform.system().lower() == "windows":
    import winsound
    winsound.PlaySound(os.path.join(os.path.dirname(__file__), "../res/snd", "DingSound.wav"),
winsound.SND_FILENAME)

GUI – Threading:

It’s the old story, if the GUI thread has to wait for your plugin, you lock up Cinema 4D – unacceptable for the user.

Doing time intensive stuff within your own threads is the simple solution to this, but keep in mind that you can’t have them call the GUI thread – instead use messages and the GeDialog’s SetTimer().

Resource reference:

Classes that use the “plugins” module need a reference to the plugins resource pointer.

This can be handled by importing those classes within the plugins *.pyp file and setting the class variable “__res__” or using a singleton (see below).

Singletons and invading Borgs:

This is something for the advanced developer – and kind of a hot topic too.

The use of globally accessible objects or ‘globals’ is often considered bad practice but also unavoidable in some cases (logger) and python even somewhat encourages it.

So if you know what you are doing and think about using a singleton, you might want to take a look at the “borg”-(anti)pattern – a monostate proxy with a twist.

class Borg:
__we_are_one = {}
    def __init__(self):       
    self.__dict__ = self.__we_are_one

Still alive

This was a lot of compressed stuff to digest, I know, but please do me the favor and don’t let any of this scare you off, it’s easier and more fun than you would expect.

I am sorry some topics where not covered in the depth they deserve –  it wasn’t easy to keep this a readable post rather than a book…

I hope this still was helpful to someone, if you have any questions, thoughts or would like me to write a more detailed post on a certain topic – let me know.

31 thoughts on “Advanced Python plugin coding for Cinema 4D”

  1. mshadis

    No specific questions as of yet but just wanted to say thank you. I upgraded to R12 a few months ago and with its built-in Python support I decided to sit down and learn Python. I\’ve been working my way through some of MIT OCW programs, etc. and am finally at a place where I know just enough to start being dangerous. Your post here was perfectly timed for me so, bravo!

  2. Focus3dD

    As expected, you did it again.
    Nice tutorial, quick and to the point.

    I have a question, though, using doc found on blender using eclipse with pydev, is it possible to launch a plugin on command line like here with blender http://www.youtube.com/watch?v=gUm1coolop0.
    I used your tip on my plugin Blee for third-party package implementation.

    Big Thanks.

  3. flashgordon Post Author

    @Focus3dD:

    Glad you liked the post – and nice job on ‘Blee’.

    I had some success with the PyDev remote debugger, but I am still investigating…

    @Jorge Arango:

    On OSX, the path to the Python interpreter should be
    ‘resource/modules/python/res/Python.osx.framework/Python’

    Also you can skip the JRE installation.

  4. ScottA

    Thanks for posting this information.
    I can now use Eclipse to create plugins.
    But Even with the c4d.py file placed in my site-packages folder. I still don’t get any auto completion.

    Any Ideas?

    -ScottA

  5. flashgordon Post Author

    Hi ScottA,

    in any case, you should get auto completion for Python – but not for the Cinema 4D API, and not within the *.pyp file (sorry – I forgot about that issue).

    However, there is a way to associate the *.pyp extension with PyDev’s Python editor – I will add it to the turotial… [Edit: done! – see point “PyDev”]

    Btw – placing the ‘c4d.py’ in the site-package folder will only enable suggestions for the symbols of the c4d package.

    Try typing “c4d.COLOR_” for example, PyDev should list you a bunch of constants.

  6. Pingback: [??? ??.13] 2011? 5? ??? | oinon is CINEMA 4D Technician.

  7. Pingback: Smart-Page.net » Blog Archive » Debugging Cinema 4D Python Plugins with PyDev

  8. VictorP

    Hello!

    Thanks for this helpfull post, I\’m actually checking python developement for C4D and I\’m an eclipse user.

    Right now I have errors every c4d related commands, even for a simple \"from c4d import plugins\".

    You say in your last comment that \"you should get auto completion for Python – but not for the Cinema 4D API\".
    So there is no way we can get completion for c4d SDK (c4d.plugins.RegisterCommandPlugin for exemple)?

    Thanks for your sharing,
    Victor

  9. flashgordon Post Author

    Hi VictorP,

    that sounds like you are trying to use PyDev to run your code – this won’t work as your interpreter misses the “c4d” module.

    Start Cinema4D instead and the import statement will work just fine.

    And I woulnd’t say there is no way to get the autocompletion running – I just haven’t figured one out yet 🙂

  10. francoisgfx

    hi,

    I’m trying to link to ‘resource/modules/python/res/Python.osx.framework/Python’

    but it tells me java.io.IOException : Cannot run program “/……./Python” : error=13, Permission denied.

    any idea ?

    cheers

  11. PeterP

    Hi!
    First of all, many thanks for this step by step tutorial.
    But there is somthing going wrong….

    Eclipse Version: Indigo Service Release 2 Build id: 20120216-1857
    Cinema: R13.051

    I created a pyDev project into my existing plugindirectory and restarted Eclipse.

    Now I got this error at the first line:
    Module: aModulName does not have a valid Python extension (it’ll not be analyzed).
    ….. Invalid python file marker

    after modifying:
    Window->Preferences->pyDev->Editor->CodeStyle->FileTypes
    Textbox Valid sourcefiles: py, pyw, pyx
    I changed to: py, pyp, pyw, pyx

    i got some more troubles:
    import c4d # ok
    import os # ok
    from c4d import plugins, utils, bitmaps # error

    Unresolved import: bitmaps

    what’s going wrong ?

  12. flashgordon Post Author

    Hi PeterP,

    thanks for pointing out the CodeStyle->FileTypes issue, I will add that to the article.

    However, I wasn’t able to reproduce the Unresolved import: bitmaps error – this is shown if PyDev was unable to find a module in your PYTHONPATH…

    When you setup the interpreter, did PyDev ask you to add a bunch of paths from /resource/modules/python/res/Python.win64.framework/ to PYTHONPATH? If you select your interpreter from the PyDev preferences, they should be listed in the “Libraries” Tab.

    When you type c4d.b, does PyDev offer you to complete to “bitmaps” ?

    Cheers!

  13. PeterP

    Thanks for your fast response!

    The bunch of paths from /resource/modules/python/res/Python.win64.framework/ is as follows:

    first item (unchecked):
    C:\Eclipse\plugins\org.python.pydev_2.4.0.2012020116\PySrc

    following items are checked:
    …\Python.win64.framework\Python26.zip
    this file doesn’t exist, there is a python26.dll!
    …\Python.win64.framework\DLLs
    …\Python.win64.framework\lib
    …\Python.win64.framework\lib\plat-win
    this path doesn’t exist!
    …\Python.win64.framework\lib\lib-tk
    …\Python.win64.framework
    …\Python.win64.framework\lib\site-packages (where the c4d.py and a c4d.pyc resides)

    I am really confused, please help!

    Thanks for your sharing,
    PeterP

  14. flashgordon Post Author

    Hm, looks ok to me…

    on the other hand it’s not really an issue, as PyDev would be unable to analyse those modules in any case.

    Still it’s strange, I never got that error, even when working with a non-C4D python interpreter…

    If you want PyDev to stop nagging you about that import, you can just add a#@UnresolvedImport comment at the end of the same line.

    This should make PyDev ignore the import statement – your code will work regardless (within C4D – NOT when run within Eclipse)

    Cheers!

  15. Marcus

    Hey!

    I’m having trouble compiling the source files on a mac.

    i’ve got a packages folder with .py class files too.

    Cinema doesn’t seem to like the pyc files generated, also is the pyp compiled, or do i need to use the source protector?

    thanks!!!

    M

    1. flashgordon Post Author

      Hi Marcus,

      The *.pyp is not to be compiled, only the *.py files.

      Did you keep the ‘__init__.pyc’ files in your package-folders and have you used a compatible (Python 2.6.4) interpreter to compile them?

      Cheers!

  16. Johannes Kollender

    Hey Jan!

    First of all, thanks a lot for the comprehensive introduction to plugin coding in Eclipse. This would have taken me ages to figure out by myself.
    Unfortunately at the moment Eclipse is still producing loads of errors on all c4d related code. For instance, if I type c4d.b (like you suggested above) the autocomplete is suggesting c4d.bitmaps but will still produce an error once it’s typed (Error message is: Undefined variable from import: bitmaps). Naturally I imported c4d earlier on.

    As far as I can tell I’ve got everything set up as you said and it’s really hard for me to figure out where things are going bad.
    Do you have any pointers at where I might be doing something wrong?

    Best
    Johannes

  17. Johannes Kollender

    Okay, by now I’ve figured out, that the real problem lies in the way I create my projects. I’ve downloaded your Smart HN-Options plugin and imported the project in Eclipse. Within that project all the syntax is recognized just fine. Even if I clean out all your files and populate the project with my own sources it works.
    I also noticed, that whenever I create a project manually the folder icon with the P in the Package Explorer has an additional little grid thingy in it, which I can’t quiet make sense of.
    I always create my projects by clicking New->PyDev Project and choose Python Grammar Version 2.6. In my version of Eclipse (4.2) I can’t uncheck anything so I choose ‘Add project directory to the PYTHONPATH?’ from the radio menu. Interpreter is left to default, but I’ve already tried setting it to Python Cinema 4D with the same dissatisfying result.
    Maybe you, as an experienced user, have an idea what I’m missing.
    Also I noticed with your Smart HN-Options plugin, that I can’t compile the sourcecode. This might be the expected behavior, because I reckon that only Cinema 4D will be able to use the files at runtime, but I just wanted to clear this up.
    I hope I’m not bothering you with this stuff, but I just learned Python 3 days ago (coming from COFFEE) and after initial success in the script editor, setting up Eclipse has been really frustrating so far.

    All the best,
    Johannes

    1. flashgordon Post Author

      Hi Johannes,

      as you might know, PyDev uses the selected Python interpreter to evaluate your code in the background.

      So if you write from c4d import bitmaps – this should already be marked as an unresolved import-error.

      Why? Because the c4d module you imported is nothing more than just the constants used by the Cinema 4D API (if you downloaded it from this article or generated it after my example) – so it will allow you to autocomplete c4d.COLOR_TEXT_BUTTON_DISABLED for example, but does not include a submodule/package called ‘bitmaps’.
      You can create a submodule or variable bitmaps for the c4d module – which would clear the import error – but as soon as you are accessing something from that pseudo-module it would be marked again.

      So in the end… until you re-create the whole API, you will have to live with some ‘invalid’ errors.
      Even if those are not really invalid as the interpreter really can’t compile or run that code without a Cinema4D instance to access – which it can’t launch from Eclipse.

      This is why you can’t just run it from within Eclipse as you correctly observed.

      I don’t know why you got a different behaviour from PyDev – maybe because the background compilation of the code takes a little time and it happens that PyDev will let you write some lines in peace, only to suddenly mark everything you just did red 🙂

      Note: you can trigger the process by saving your document – which will also save a backup-copy in the history of the file so… “save early, save often”.

      What i guess is happening with my project is, that somehow it got imported wrong and you now have only basic syntax highlighting but no auto-completion at all – which I prefer to be honest – although having the interpreter back you up is nice too.

      Hope this answers your questions and you have a smooth transition to working with Python

      Cheers

      – Jan

  18. Johannes Kollender

    Hi Jan!

    Thanks for the support. I’ve finally figured out, that the errors only occurred, whenever I had ‘Add project directory to the PYTHONPATH?’ enabled. For some reason the interpreter got confused when a source folder was set. This is the way your project file from the Smart HN-Options plugin was set up, so it was working fine all along.
    Now that I got this figured out things are going really smoothly. Thanks again for your detailed reply and of course your fantastic tutorial.
    First plugin is almost ready and I’m a happy man. 🙂

    Looking forward to any upcoming tutorials on your site.

    All the best,
    Johannes

    1. flashgordon Post Author

      Interesting observation – thank you for mentioning that.
      Glad you are a happy with your coding environment now.

      Sadly I am quite busy atm but there will be new tutorials – I think the next one is gonna tackle threading… we’ll see 🙂

      Cheers!

  19. Pingback: Scripting Resources for Motion Graphics | LESTERBANKS

    1. flashgordon Post Author

      Hi fanhang,

      well, you can’t directly ‘connect’ the two, but you should be able to setup a nice workflow with the PyCharm IDE:

      1. Install the Python 2.6.4 interpreter.
      2. Download the C4D API fake module (http://www.smart-page.net/python/c4d_fake_package.zip) and place it in the ‘site-packages’ directory of that interpreter.
      3. Configure PyCharm to use that interpreter (https://www.jetbrains.com/pycharm/quickstart/configuring_interpreter.html)
      4. Make the *.pyp extension known to PyCharm’s editor (https://www.jetbrains.com/pycharm/help/register-new-file-type-association-dialog.html)
      5. Create a new project within the ‘/Plugins’ directory of your C4D installation and enjoy your new IDE 🙂

      PyCharm also features a remote debugger.

      Cheers!

      – Jan

Leave a Comment

Your email address will not be published.