Learn About Using Threads Inside Python For A Windows Delphi App In This Sample

We know Delphi supports Multithreading. Multithreading in Python can be achieved using Python Module Threading. However, In a use case like Delphi Application embedding Python(Python4Delphi) or CPython, the interpreter is not fully thread-safe. In order to support multi-threaded Python programs, there’s a global lock, called the global interpreter lock or GIL, that must be held by the current thread before it can safely access Python objects. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.

Some extension modules, either standard or third-party, are designed so as to release the GIL when doing computationally-intensive tasks such as compression or hashing. Also, the GIL is always released when doing I/O. More Details here. This post will guide you on how to evaluate several python functions concurrently using Python4Delphi TPyDelphiThread.

Python4Delphi Demo11 Sample App shows how to achieve concurrency(using more interpreters) inside Python. You can find the Demo11 source on GitHub.

Prerequisites: Download and install the latest Python for your platform. Follow the Python4Delphi installation instructions mentioned here. Alternatively, you can check out this video Getting started with Python4Delphi.

Components used in Python4Delphi Demo11 App:

• TPythonEngine: A collection of relatively low-level routines for communicating with Python, creating Python types in Delphi, etc. It’s a singleton class.
• TPythonModule: It’s inherited from TMethodsContainer class allows creating modules by providing a name. You can use routines AddMethod, AddMethodWithKW to add a method of type PyCFunction. You can create events using the Events property.
• TPaintBox provides a canvas that applications can use for rendering an image.
• TPyDelphiThread: Inherited from TThread has properties like ThreadState( A pointer which stores Python last state), ThreadExecMode(emNewState, emNewInterpreter). Protected functions like ExecuteWithPython, Py_Begin_Allow_Threads, Py_End_Allow_Threads helps to run concurrently without thread conflicts.
• TMemo: A multiline text editing control, providing text scrolling. The text in the memo control can be edited as a whole or line by line.

You can find the Python4Delphi Demo11 sample project from the extracted GitHub repository ..Python4DelphiDemosDemo11.dproj. Open this project in RAD Studio 10.4.1 and run the application.

Implementation Details:

• PythonEngine component provides the connection to Python or rather the Python API. This project uses Python3.9 which can be seen in TPythonEngine DllName property.
• SortModule(TPythonModule) has initialized with 2 Delphi Methods SortModule_GetValue, SortModule_Swap which is imported in python script to perform sorting. 3 arrays are randomized with integer values, later get sorted. Three Sort functions were defined in the script such as BubbleSort, SelectionSort, and QuickSort which is evaluated by PyDelphiThread Instance’s ExecuteWithPython procedure. Note: Don’t override Execute Method, use always ExecuteWithPython.

In this Sample, one interpreter Button uses an emNewState(single interpreter with new state and upon execution completion, restores the thread state) ThreadExecMode and three interpreter button use an emNewInterpreter (same as a new state but with new interpreter fully initialized) ThreadExecMode to Execute.

procedure TThreadSortForm.InitThreads(ThreadExecMode: TThreadExecMode; script: TStrings);
begin
  RandomizeArrays;
  ThreadsRunning := 3;
  with GetPythonEngine do
  begin
    OwnThreadState := PyEval_SaveThread;

    with TSortThread.Create( ThreadExecMode, script, SortModule, 'SortFunc1',
                             BubbleSortBox, BubbleSortArray) do
      OnTerminate := ThreadDone;
    with TSortThread.Create( ThreadExecMode, script, SortModule, 'SortFunc2',
                                SelectionSortBox, SelectionSortArray) do
      OnTerminate := ThreadDone;
    with TSortThread.Create( ThreadExecMode, script, SortModule, 'SortFunc3',
                            QuickSortBox, QuickSortArray) do
      OnTerminate := ThreadDone;
  end;

  StartBtn.Enabled := False;
  Start2Btn.Enabled := False;
end;

When all the thread is executed the OwnThreadState is restored as shown below.

procedure TThreadSortForm.ThreadDone(Sender: TObject);
begin
  Dec(ThreadsRunning);
  if ThreadsRunning = 0 then
  begin
    GetPythonEngine.PyEval_RestoreThread(OwnThreadState);
    StartBtn.Enabled := True;
    Start2Btn.Enabled := True;
    ArraysRandom := False;
  end;
end;

On Clicking one interpreter or three interpreter button, the three thread is initialized and executes python script by the TSortThread.ExecuteWithPython procedure as shown below.

procedure TSortThread.ExecuteWithPython;
var pyfunc: PPyObject;
begin
  with GetPythonEngine do
  begin
    if Assigned(FModule) and (ThreadExecMode = emNewInterpreter) then
      FModule.InitializeForNewInterpreter;
    if Assigned(fScript) then
      ExecStrings(fScript);
    pyfunc :=  FindFunction( ExecModule, fpyfuncname);
    if Assigned(pyfunc) then
      try
        EvalFunction(pyfunc,[NativeInt(self),0,FSize]);
      finally
        Py_DecRef(pyfunc);
      end;
  end;
end;

Upon execution the 3 PaintBox’s is get updated by the TSortThread.VisualSwap procedure.

Hope this helps to understand threading inside python for a Delphi Application.

Head over and find out more about building Windows Apps with Python4Delphi on GitHub.