Error running salomeTools when salome environment is loaded

Hello.

I use Kubuntu 18.04 and I am trying to use salomeTools. I have just downloaded SALOME-9.8.0-UB18.04-SRC.tar.gz, unpacked it and tried the following command:

./sat template --name test --template CppComponent --target ./as

It runs well and I get the following output:

create log dir /home/emilio/Codigos/SALOME-9.8.0-UB18.04-SRC/LOGS
Create sources from template
  destination = ./as/test 
  name = test 
  template = CppComponent 

Name of the module? [Test] 
resources directory (lowercase)? [test] 
get method for prepare? [archive] 

Create configuration file: ./as/test/test.pyconf
The sources were created in ./as/test
Do not forget to put them in your version control system.

However, if before running this command I load the SALOME environment using source env_launch.sh I get an error message. This is the output I get in the terminal:

create log dir /home/emilio/Codigos/SALOME-9.8.0-UB18.04-SRC/LOGS
Create sources from template
  destination = ./as/test 
  name = test 
  template = CppComponent 

Name of the module? [Test] 
resources directory (lowercase)? [test] 
get method for prepare? [archive] 


***** salomeTools ERROR: sat template

Traceback (most recent call last):
  File "/home/emilio/Codigos/SALOME-9.8.0-UB18.04-SRC/salomeTools/src/salomeTools.py", line 478, in run_command
    res = __module__.run(argv, self, logger_command)
  File "/home/emilio/Codigos/SALOME-9.8.0-UB18.04-SRC/salomeTools/commands/template.py", line 522, in run
    target_dir, conf_values, logger)
  File "/home/emilio/Codigos/SALOME-9.8.0-UB18.04-SRC/salomeTools/commands/template.py", line 317, in prepare_from_template
    m = f.read()
  File "/home/emilio/Codigos/SALOME-9.8.0-UB18.04-SRC/BINARIES-UB18.04/Python/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

How can I run salomeTools with the SALOME environment loaded? Is this the normal default behaviour? Thank you in advance.

The error happens while reading a png file from the resources folder, in file template.py of salomeTools. The error appears because a png file is not a text file but a binary one, and the script attempts to read it as a text file.

I can workaround the problem using the following patch. EDIT: a proper patch is given in the third message of this thread.

--- template.py 2022-03-16 15:20:15.000588031 +0100
+++ template.py.old     2022-03-16 15:18:46.961717697 +0100
@@ -313,12 +313,11 @@
                 logger.write("  - %s\n" % fpath[pathlen:], 5)
                 continue
             # read the file
-            if not fpath.lower().endswith('.png'):
-                with open(fpath, 'r') as f:
-                    m = f.read()
-                    # make the substitution
-                    template = CompoTemplate(m)
-                    d = template.safe_substitute(dico)
+            with open(fpath, 'r') as f:
+                m = f.read()
+                # make the substitution
+                template = CompoTemplate(m)
+                d = template.safe_substitute(dico)
                         
             changed = " "
             if d != m:

I am not sure if it has side-effects, but I hope it helps you. I think the source of the problem is at line 312:

             if not tsettings.check_file_for_substitution(fpath[pathlen:]):

EDIT: the proper fix is given in the third message of this thread.

I have been doing some more tests. When I do not source Salome, in my system the command python points to the Python 2 version installed in the system and it works. However, when I source Salome, the command python points to the Python 3 version shipped with Salome and it fails.

I have also tried SalomeTools with the python3 version of my system, not sourcing Salome, and it also fails, so it is something related to using Python 3.

It seems that the map function used to return a list in Python 2 but now in Python3 it returns an iterator that can be used only once:

https://docs.python.org/3.0/whatsnew/3.0.html#views-and-iterators-instead-of-lists

The following patch includes a proper fix for the bug:

--- template.py.old     2021-12-20 20:56:38.000000000 +0100
+++ template.py.new     2022-03-17 12:55:22.050165656 +0100
@@ -123,8 +123,7 @@
         self.delimiter_char = get_dico_param(ldic, "delimiter", ":sat:")
 
         # get the ignore filter
-        self.ignore_filters = map(lambda l: l.strip(),
-                                  ldic["ignore_filters"].split(','))
+        self.ignore_filters = [l.strip() for l in ldic["ignore_filters"].split(',')]
 
     def has_pyconf(self):
         return len(self.pyconf) > 0

Dear Emilio
Thanks a lot for the feedback, the bug you reported and the patch to correct it.
We will integrate it.

Indeed we never source salome when using sat, so most of the time sat is still used under python 2.
There remains some python3 related bugs in the parts not frequently used (sat template being one of them).

Thanks again.
Best Regards