[x3d-public] Generating python issue. Windows 10

[John Carlson]

Subject: "X3dToJson.xslt emits non-UTF-8 JSON????"

in x3d-public,

indicated that this problem may be with cp1252 generation from Java in a
Saxon stylesheet is not overridden with utf-8 generation.  On windows 10.

Not sure why?
Ss

On Sat, Jul 2, 2022 at 1:24 AM John Carlson <yottzumm at gmail.com> wrote:

> Here's a patch that appears to work on Windows. Other systems are welcome
> to try something else!
> $ git diff .
> diff --git a/src/main/python/x3djsonld.py b/src/main/python/x3djsonld.py
> index 184f21fd7..4dd770f68 100644
> --- a/src/main/python/x3djsonld.py
> +++ b/src/main/python/x3djsonld.py
> @@ -256,7 +256,7 @@ def parseObject(parent, data,indent):
>      return out
>
>
> -with open('HelloWorldProgramOutput.json' if len(sys.argv) == 1 else
> sys.argv[1], encoding="utf-8") as json_file:
> +with open('HelloWorldProgramOutput.json' if len(sys.argv) == 1 else
> sys.argv[1], encoding="cp1252") as json_file:
>      data = json.load(json_file)
>      print("from x3d import *")
>      # print("from x3d import SFBool")
>
>
> Advice on what encoding to use welcome for cross-platform???  latin1?
>  The greek letter for pi actually appears when I cat the json file.
>
> On Sat, Jul 2, 2022 at 12:32 AM John Carlson <yottzumm at gmail.com> wrote:
>
>> Python problem. Output to terminal is not the same as output to pipe.
>>
>> Git Bash on Windows 10.  Also under PowerShell. Under Windows Terminal.
>> Python 3.10.2.
>>
>> I believe if I remove the pi Unicode character it works. JSON attached.
>>
>> Code is here at this URL:
>>
>>
>> https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/python/x3djsonld.py
>>
>> This is an encoding issue AFAIK.  Probably we've handled it before.
>>
>> See below:
>> $ find /c/x3d-code/
>> www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter02GeometryPrimitives/
>> -name TextSpecialCharacters.json| xargs python3 x3djsonld.py
>> ==================================================
>> '''
>> x3d.py package 4.0.64.1 loaded, have fun with X3D Graphics!
>> '''
>> from x3d import *
>> print(
>> X3D(profile="Immersive", version="3.3",
>> head=head(),
>> Scene=Scene(children=[
>> WorldInfo(title="TextSpecialCharacters.x3d"),
>> Background(skyColor=[(1,1,1)]),
>> Viewpoint(description="Default View", position=((0,0,15))),
>> Shape(
>> geometry=
>> Text(DEF="DefaultText", string=["Character entity substitutions:","empty
>> string \"\" skips a line:","","apostrophe ' is '","ampersand & is
>> &","quote mark \" is "","backslash \\ is X3D escape character","double
>> backslash \\\\ is X3D backslash \\ character","Pi Π is Π XML character
>> entity"],
>> fontStyle=
>> FontStyle(DEF="CenteredFontStyle", justify=["MIDDLE","MIDDLE"])),
>> appearance=
>> Appearance(
>> material=
>> Material(DEF="DefaultMaterial", diffuseColor=((0.2,0.2,0.2)))))]))
>> .XML())
>> =====================================================
>> $ find /c/x3d-code/
>> www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter02GeometryPrimitives/
>> -name TextSpecialCharacters.json| xargs python3 x3djsonld.py | cat
>> =====================================================
>> '''
>> x3d.py package 4.0.64.1 loaded, have fun with X3D Graphics!
>> Traceback (most recent call last):
>>   File "x3djsonld.py", line 265, in <module>
>> '''
>> from x3d import *
>> print(    print(parseObject("X3D", data, 0))
>>   File "C:\Program
>> Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\encodings\cp1252.py",
>> line 19, in encode
>>     return codecs.charmap_encode(input,self.errors,encoding_table)[0]
>> UnicodeEncodeError: 'charmap' codec can't encode character '\u03a0' in
>> position 513: character maps to <undefined>
>>
>>
>> Tried under PowerShell for Microsoft people. I don't see why there's so
>> much repetition in errors.
>> =====================================================
>> PS C:\Users\john\X3DJSONLD\src\main\python> python .\x3djsonld.py
>> ..\data\TextSpecialCharacters.json| Get-Content
>> ======================================================
>> Traceback (most recent call last):
>>   File "C:\Users\john\X3DJSONLD\src\main\python\x3djsonld.py", line 265,
>> in <module>
>>     print(parseObject("X3D", data, 0))
>>   File "C:\Python310\lib\encodings\cp1252.py", line 19, in encode
>>     return codecs.charmap_encode(input,self.errors,encoding_table)[0]
>> UnicodeEncodeError: 'charmap' codec can't encode character '\u03a0' in
>> position 470: character maps to <undefined>
>> Get-Content : The input object cannot be bound to any parameters for the
>> command either because the
>> command does not take pipeline input or the input and its properties do
>> not match any of the
>> parameters that take pipeline input.
>> At line:1 char:59
>> + python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content
>> +                                                           ~~~~~~~~~~~
>>     + CategoryInfo          : InvalidArgument: (''':PSObject)
>> [Get-Content], ParameterBindingExcepti
>>    on
>>     + FullyQualifiedErrorId :
>> InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand
>>
>> Get-Content : The input object cannot be bound to any parameters for the
>> command either because the
>> command does not take pipeline input or the input and its properties do
>> not match any of the
>> parameters that take pipeline input.
>> At line:1 char:59
>> + python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content
>> +                                                           ~~~~~~~~~~~
>>     + CategoryInfo          : InvalidArgument: (x3d.py package ...h X3D
>> Graphics!:PSObject) [Get-Con
>>    tent], ParameterBindingException
>>     + FullyQualifiedErrorId :
>> InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand
>>
>> Get-Content : The input object cannot be bound to any parameters for the
>> command either because the
>> command does not take pipeline input or the input and its properties do
>> not match any of the
>> parameters that take pipeline input.
>> At line:1 char:59
>> + python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content
>> +                                                           ~~~~~~~~~~~
>>     + CategoryInfo          : InvalidArgument: (''':PSObject)
>> [Get-Content], ParameterBindingExcepti
>>    on
>>     + FullyQualifiedErrorId :
>> InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand
>>
>> Get-Content : The input object cannot be bound to any parameters for the
>> command either because the
>> command does not take pipeline input or the input and its properties do
>> not match any of the
>> parameters that take pipeline input.
>> At line:1 char:59
>> + python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content
>> +                                                           ~~~~~~~~~~~
>>     + CategoryInfo          : InvalidArgument: (from x3d import
>> *:PSObject) [Get-Content], Parameter
>>    BindingException
>>     + FullyQualifiedErrorId :
>> InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand
>>
>> Get-Content : The input object cannot be bound to any parameters for the
>> command either because the
>> command does not take pipeline input or the input and its properties do
>> not match any of the
>> parameters that take pipeline input.
>> At line:1 char:59
>> + python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content
>> +                                                           ~~~~~~~~~~~
>>     + CategoryInfo          : InvalidArgument: (print(:PSObject)
>> [Get-Content], ParameterBindingExce
>>    ption
>>     + FullyQualifiedErrorId :
>> InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220705/9aa992ca/attachment-0001.html>