Computing: Website and Database Programming

Web development environment setup on MS Windows.


6. Python.
  6.1. Python distributions.
   
"Python is a programming language that lets you work quickly and integrate systems more effectively. Whether you're new to programming or an experienced developer, it's easy to learn and use Python."
The paragraph above is how the Python Software Foundation describes their product on the Python website. Python is commonly used in the following domains: Web and Internet development, scientific and numeric computing, education, desktop GUI development (thanks to several freely available GUI libraries), software development, and business applications.
Python for Windows may be downloaded free of charge from the Python website. You'll find also there distributions for macOS and Linux. Note, that there are two versions of Python: Python 2 and Python 3. Even though these are just different releases of the same programming language, there are striking differences between the two. Python 2 code will not work with the Python 3 interpreter and Python 3 code will not work with the Python 2 interpreter. This is in particular true for Python modules, that have to be Python 3 code to be run by a Python 3 interpreter. The version used in this tutorial is Python 3.10.6 (x64). My actual operating system is Windows 10, 64bit.
  6.2. Installing Python for Windows.
   
The file, downloaded from the Python website (called python-3.10.6-amd64.exe in my case) is an installer program with a wizard that guides you through the setup process. I chose to use the default installation, that installs the application components (Python, IDLE, pip and documentation) into the users AppData folder, creates Start menu shortcuts and file associations and, the corresponding checkbox being selected, installs the launcher for all users. If you intend to run Python from Command Prompt, be sure to select the Add Python 3.10 to path checkbox.
Installing Python 3 on MS Windows
When the installation is done, you'll have the possibility to disable the 260-characters path length limit (for Python and other applications). No reason not to do it, I think.
  6.3. Running a Python console program.
   
The Python 3 default installation includes a Python GUI shell (IDLE) as well as a Python command line shell; with IDLE installed, this latter one becomes more or less useless. Any Python code entered in the main IDLE window will be immediately executed when you hit the ENTER key, as you can see in the upper part of the following screenshot. In IDLE, you can also create new Python files or open existing ones. In this case, a second window (an editor, in fact) opens. The code entered in the editor window can be executed (interpreted by Python 3) using the menu command Run > Run module. The middle part of the screenshot shows the editor window with the same Python code, that I executed before in the shell, and that I saved as "hello.py" into the "Python" folder of my Programming library.
'Hello world' script in the Python GUI shell and editor IDLE
You can run Python script files in Windows Command Prompt. As the location of the Python interpreter has been added to the PATH environment variable, we can invoke Python without having to specify a path. Concerning the .py file, either specify the full path, or (as done in the following screenshot) cd to the directory where it is located. Supposing to be in the directory, where I saved my hello.py file, the script may be run in Command Prompt by typing:
    python hello.py
Running the 'Hello world' script in Windows Command Prompt
  6.4. Thonny: A Python IDE for beginners.
   
IDLE allows well to write and execute Python scripts, but a "real" IDE can make life lots easier. There are several IDEs for Python available. I actually use the freeware application Thonny, that can be downloaded from the Thonny website. It's the developers themselves who call Thonny a Python IDE for beginners. Available for Windows, macOS and Linux, it is simple to use and includes all features that a beginner needs to comfortably develop Python scripts. Also to note, that Thonny is bundled with some (older) Python distribution, in the case of Thonny 3.3.13 with Python 3.7.9. Thus, if you only intend to give Python a little try, you could skip the installation of the latest Python distribution and only install Thonny...
You can install as many versions of Python on your computer as you want. By default, each of them is installed into its own PythonNMM folder (folder Python310 for Python 3.10.x for example). To note that if you install a new version (that you want to use by default) beside one or several existing ones and if you intend to run Python in Command Prompt, be sure to change the PATH environment variable. In Thonny, you can choose the version of Python, that you want to use. That's what we'll have to do here, too, because Thonny uses by default the bundled 3.7.9 version and we'd like to use our newer 3.10.6 version. From the menu, choose Tools > Options and select the Interpreter tab. As interpreter or device to use for running the code within Thonny, choose Alternative Python 3 interpreter or virtual environment. Open the drop-down list for possible values as Python executable and choose Python 3.10.6 (i.e. the file python.exe located in the Python310 folder in ...\AppData\...).
Thonny: Defining the Python interpreter to be used (by default)
  6.5. Running Python CGI on Apache.
   
There are several ways to run Python scripts on Apache, the best way to do on a real-life Internet site being probably using Web Server Gate Interface (WSGI), that allows improved performance and great scalability, whereas Common Gate Interface (CGI) becomes slower and slower when the number of requests increases. On our local webserver, nothing speaks against the usage of CGI and it's the only method described in this tutorial. For details concerning WSGI, please search the Internet.
Here's how I described CGI in my tutorial about Perl CGI: CGI is a set of standards that defines how information is exchanged between a web server and a custom script. Normally, when a browser contacts a web server using the HTTP protocol to ask for a URL (i.e. a web page filename), the server will check the URL and will look for the filename requested. If it finds the file, it sends it back to the browser without any further execution (or, if it contains code such as a PHP script, lets execute the PHP interpreter this code); if it doesn't find the file, it sends an error message (Error 404).
However, it is possible to set up the HTTP server in such a way, so that whenever a file in a certain directory is requested, that file is not sent back; instead it is executed as a program, and whatever that program outputs as a result, that is sent back to your browser to display. This can be done by using CGI, a special functionality available within the web server. Programs, that are executed by the server to produce the final result (mostly creation of a webpage), are called CGI scripts. These CGI programs can be Perl or Python scripts, shell scripts, C or C++ programs, etc.
Running a Python CGI script on Apache needs 2 things:
  • Configuring a script directory in the Apache configuration file (cf. part 1).
  • Using the shebang (#!) line as first line in the CGI script to inform the web server where the Python interpreter is located.
Notes.
  • It is possible to run CGI scripts from any directory by setting a handler and using the ExecCGI option in the Options directive. This is however not recommended for security and other reasons.
  • On Linux, the shebang line is mandatory for all scripts. On MS Windows, it may be omitted for command line programs, but you always must specify it in a CGI script.
  • If the Apache web server runs on Linux (as it does on most web hosting Internet sites), the CGI script has to be set executable. A chmod 0755 works well.
Configuring Apache for CGI.
If you have configured Apache for Perl, as described in the Web development environment setup on MS Windows: 5. Perl, you can skip the following paragraphs: All configuration is done and, placing your Python CGI script in the cgi-bin folder, it should work correctly without that you have to change anything on Apache.
If you have installed Apache in its default directory, you normally haven't to make any modifications to the httpd.conf file, as Apache is distributed configured to use its CGI features (to be sure, check if they are not commented out!). As I installed my web server to "C:\Programs" (cf. part 1), the CGI related entries in httpd.conf have to be adapted as follows:
    ScriptAlias /cgi-bin/ "c:/Programs/Apache24/cgi-bin/"
    <Directory "c:/Programs/Apache24/cgi-bin">
        AllowOverride None
        Options None
        Require all granted
    </Directory>
Errors in a CGI script may result in an Internal server error, with display of an "Error 500" error page. To get more information about the reason of this error, having a look at the Apache error file normally helps.
Testing Python CGI.
The following two screenshots show the code of a simple "Hello World" Python CGI script opened in Thonny (left), script that I saved as hello.cgi.py in my webserver's cgi-bin directory and the execution of the script in the Firefox web browser (right).
Simple 'Hello World' Python CGI script opened in Thonny
Execution of a 'Hello World' Python CGI script in Firefox
  6.6. Installing Python packages using PIP.
   
As for Perl, Python code libraries are called modules. Modules, that are not part of the Python standard library are installed as packages, that are nothing else but all of the files, you need, to use a given module. In Perl, modules are made available to a program script by naming them in the use statement. Similarly, in Python, modules are included into the actual namespace by naming them in the import statement.
You can install Python packages manually, but this is not such a good idea. Because you would have to take care yourself of the two important things when installing packages: 1. In order not to mess up your Python installation, you have to install the correct package version for the version of Python, you are actually running. 2. Packages may have dependencies (i.e. need other packages to run correctly), that must be installed with the package, you want to install. Instead of installing packages manually, all interest in doing it by using a package manager. The Python package manager is called Python PIP (or PIP for short) and has been installed by default with the Python 3 Windows installer, that we downloaded and installed at the beginning of this tutorial.
PIP may be run as a module in Command Prompt. If you enter
    python -m pip
PIP usage information with its parameters and options will be displayed.
Installing a Python package with PIP is simple (you must know the module name, of course):
    python -m pip install {package-name}
The following screenshot shows, how PIP can be used to install the package PyMySQL, that is needed to connect with Python to a MySQL database (cf. tutorial part 6.7).
Python PIP: Installing the package PyMySQL
  6.7. Accessing MySQL databases with Python.
   
The Python standard for database interfaces is the Python DB-API. Most Python database interfaces adhere to this standard. Python Database API supports a wide range of database servers, including MySQL, MariaDB, PostgreSQL and Microsoft SQL Server. You need a separate DB API module for each database you need to access. The DB API provides a minimal standard for working with databases using Python structures and syntax wherever possible. This API includes the following:
  • Importing the API module.
  • Acquiring a connection with the database.
  • Issuing SQL statements and stored procedures.
  • Closing the connection.
Python has an in-built support for SQLite. Concerning MySQL, there is a popular module called MySQLdb; however this module is not compatible with Python 3. Instead, we shall use the PyMySQL module. PyMySQL is an interface for connecting to a MySQL database server from Python implementing the Python Database API v2.0 and containing a pure-Python MySQL client library. The goal of PyMySQL is to be a drop-in replacement for MySQLdb.
To check if PyMySQL is installed within the Python 3 installation, just enter
    import pymysql
in the Python shell (IDLE, or shell included within Thonny). As PyMySQL is not included within the Python 3 for Windows installer, you will get an error message, as shown on the following screenshot.
Using the Python shell of Thonny to check if PyMySQL is installed
As we saw above, Python packages may be installed using the Python package manager PIP. In Windows Command Prompt, enter (cf. screenshot in part 6.6):
    python -m pip install PyMySQL
Simple MySQL (CGI) script.
The following script does the same as we did with PHP in part 3 and with Perl in part 5: Displaying the number of cities in the "city" table of the "world" database (cf. part 2 of the tutorial).
    #!C:/Users/allu/AppData/Local/Programs/Python/Python310/python.exe
    import pymysql
    print("Content-type: text/html")
    print()
    print("<html>")
    print("<head><title>Python MySQL test</title></head>")
    print("<body>")
    # Connect to database 'world'
    db = pymysql.connect(host='localhost', user='nemo', password='nemo', db='world')
    cursor = db.cursor()
    # Read the number of records in the 'city' table
    cursor.execute("SELECT count(*) FROM city")
    count = cursor.fetchone()[0]
    # Print result onto webpage
    print("<p>Number of cities in database 'world' = ", count, "</p>")
    print("</body>")
    print("</html>")
    # Disconnect from database
    db.close()
There is no reason not to run a CGI script in the console for testing purposes. Doing so in Thonny allows us to view the output of Thonny Assistant (if the Assistant window isn't shown, select it in View): errors, warnings and suggestions how we possibly could resolve the issues.
Running a Python CGI script in Thonny for testing purposes
The funny thing with this tutorial is that I never wrote a Python program in my whole life, thus, I'm a complete Python newbie with a Python knowledge of not lots more than none. To write the little scripts for the tutorial, I searched the Internet for similar examples, that I modified as needed. Some errors during testing and at a certain moment Thonny Assistant suggested to install the types-PyMySQL package. As I did not import this module into the namespace of my script, it's not needed here, but, maybe with other MySQL related operations. Install it or not...
Some remarks concerning the code of my Python-MySQL CGI script:
  • The shebang line, indicating the path to the Python 3 interpreter, is mandatory and must be the first line of the script. On Windows, you may either use the slash or the backslash as directory separator in the path string.
  • As PyMySQL is not part of the standard Python 3 library, you'll have to import the pymysql module.
  • Creating the result webpage is the same as in Perl: Writing a website header (using the statement print("Content-type: text/html") and then using print() to write the HTML code.
  • Accessing the MySQL database has a rather different syntax as in Perl, in particular, you'll have to create a cursor and use this object's methods to communicate with the MySQL server (in Perl, you can directly use the methods of the database handle created with the DBI->connect method.
  • In several tutorials and examples, that I found on the web, the connection to the MySQL statement was implemented with the code
        db = PyMySQL.connect("hostname","username","password","database")
    This did not work with my Python 3 installation. To be sure, use named parameters instead of positional parameters for the pymysql.connect() method:
        db = pymysql.connect(host='hostname', user='username', password='password', db='database')
  • Applying indexes directly to a function call is something I commonly do in Free Pascal. In Python, this seems not really be supported, but seems to work correctly (cf. Thonny Assistant warning in the screenshot above). A more correct code would be to apply the index to the "count" variable:
        print("<p>Number of cities in database 'world' = ", count[0], "</p>").
To terminate the tutorial, a screenshot of the output of the CGI script mysql.py (placed in the cgi-bin directory) in Firefox web browser:
Python-MySQL CGI script output in Firefox web browser

If you find this text helpful, please, support me and this website by signing my guestbook.