<div dir="ltr"><h2 id="gmail-:1e5" class="gmail-hP" tabindex="-1" style="margin:0px;padding:0px 10px 0px 0px;border:0px;font-variant-ligatures:no-contextual;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-weight:400;font-stretch:inherit;font-size:1.375rem;line-height:inherit;font-family:"Google Sans",Roboto,RobotoDraft,Helvetica,Arial,sans-serif;display:inline;outline:none;word-break:break-word;color:rgb(32,33,36)">Subject: "X3dToJson.xslt emits non-UTF-8 JSON????"</h2><div><font color="#202124" face="Google Sans, Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:22px;font-variant-ligatures:no-contextual"><br></span></font></div><div><font color="#202124" face="Google Sans, Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:22px;font-variant-ligatures:no-contextual">in x3d-public,</span></font></div><div><font color="#202124" face="Google Sans, Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:22px;font-variant-ligatures:no-contextual"><br></span></font></div><div><font color="#202124" face="Google Sans, Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:22px;font-variant-ligatures:no-contextual">indicated that this problem may be with cp1252 generation from Java in a Saxon stylesheet is not overridden with utf-8 generation. On windows 10.</span></font></div><div><font color="#202124" face="Google Sans, Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:22px;font-variant-ligatures:no-contextual"><br></span></font></div><div><font color="#202124" face="Google Sans, Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:22px;font-variant-ligatures:no-contextual">Not sure why?<br></span></font><div><span id="gmail-:1wd" class="gmail-J-J5-Ji" style="display:inline-flex;min-height:28px;vertical-align:bottom;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:22px"><div class="gmail-pG" aria-label="Not important" role="img" id="gmail-:1tn" style="font-size:0px;padding:0px;width:20px;height:20px;display:inline-block;margin:0px 0px 0px 8px">Ss</div></span></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jul 2, 2022 at 1:24 AM John Carlson <<a href="mailto:yottzumm@gmail.com">yottzumm@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Here's a patch that appears to work on Windows. Other systems are welcome to try something else!<div>$ git diff .<br>diff --git a/src/main/python/x3djsonld.py b/src/main/python/x3djsonld.py<br>index 184f21fd7..4dd770f68 100644<br>--- a/src/main/python/x3djsonld.py<br>+++ b/src/main/python/x3djsonld.py<br>@@ -256,7 +256,7 @@ def parseObject(parent, data,indent):<br> return out<br><br><br>-with open('HelloWorldProgramOutput.json' if len(sys.argv) == 1 else sys.argv[1], encoding="utf-8") as json_file:<br>+with open('HelloWorldProgramOutput.json' if len(sys.argv) == 1 else sys.argv[1], encoding="cp1252") as json_file:<br> data = json.load(json_file)<br> print("from x3d import *")<br> # print("from x3d import SFBool")<br></div><div><br></div><div><br></div><div>Advice on what encoding to use welcome for cross-platform??? latin1? The greek letter for pi actually appears when I cat the json file.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jul 2, 2022 at 12:32 AM John Carlson <<a href="mailto:yottzumm@gmail.com" target="_blank">yottzumm@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Python problem. Output to terminal is not the same as output to pipe.<div><br></div><div>Git Bash on Windows 10. Also under PowerShell. Under Windows Terminal. Python 3.10.2.</div><div><br></div><div>I believe if I remove the pi Unicode character it works. JSON attached.</div><div><br></div><div>Code is here at this URL:</div><div><br></div><div><a href="https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/python/x3djsonld.py" target="_blank">https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/python/x3djsonld.py</a><br></div><div><br></div><div>This is an encoding issue AFAIK. Probably we've handled it before.</div><div><br></div><div>See below:</div><div>$ find /c/x3d-code/<a href="http://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter02GeometryPrimitives/" target="_blank">www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter02GeometryPrimitives/</a> -name TextSpecialCharacters.json| xargs python3 x3djsonld.py<div>==================================================<br>'''<br>x3d.py package 4.0.64.1 loaded, have fun with X3D Graphics!<br>'''<br>from x3d import *<br>print(<br>X3D(profile="Immersive", version="3.3",<br>head=head(),<br>Scene=Scene(children=[<br>WorldInfo(title="TextSpecialCharacters.x3d"),<br>Background(skyColor=[(1,1,1)]),<br>Viewpoint(description="Default View", position=((0,0,15))),<br>Shape(<br>geometry=<br>Text(DEF="DefaultText", string=["Character entity substitutions:","empty string \"\" skips a line:","","apostrophe ' is '","ampersand & is &","quote mark \" is "","backslash \\ is X3D escape character","double backslash \\\\ is X3D backslash \\ character","Pi Π is Π XML character entity"],<br>fontStyle=<br>FontStyle(DEF="CenteredFontStyle", justify=["MIDDLE","MIDDLE"])),<br>appearance=<br>Appearance(<br>material=<br>Material(DEF="DefaultMaterial", diffuseColor=((0.2,0.2,0.2)))))]))<br>.XML())<br>=====================================================</div><div>$ find /c/x3d-code/<a href="http://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter02GeometryPrimitives/" target="_blank">www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter02GeometryPrimitives/</a> -name TextSpecialCharacters.json| xargs python3 x3djsonld.py | cat</div><div>=====================================================<br>'''<br>x3d.py package 4.0.64.1 loaded, have fun with X3D Graphics!<br>Traceback (most recent call last):<br> File "x3djsonld.py", line 265, in <module><br>'''<br>from x3d import *<br>print( print(parseObject("X3D", data, 0))<br> File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\encodings\cp1252.py", line 19, in encode<br> return codecs.charmap_encode(input,self.errors,encoding_table)[0]<br>UnicodeEncodeError: 'charmap' codec can't encode character '\u03a0' in position 513: character maps to <undefined><br></div></div><div><br></div><div><br></div><div>Tried under PowerShell for Microsoft people. I don't see why there's so much repetition in errors.</div><div>=====================================================</div><div>PS C:\Users\john\X3DJSONLD\src\main\python> python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content</div><div>======================================================<br>Traceback (most recent call last):<br> File "C:\Users\john\X3DJSONLD\src\main\python\x3djsonld.py", line 265, in <module><br> print(parseObject("X3D", data, 0))<br> File "C:\Python310\lib\encodings\cp1252.py", line 19, in encode<br> return codecs.charmap_encode(input,self.errors,encoding_table)[0]<br>UnicodeEncodeError: 'charmap' codec can't encode character '\u03a0' in position 470: character maps to <undefined><br>Get-Content : The input object cannot be bound to any parameters for the command either because the<br>command does not take pipeline input or the input and its properties do not match any of the<br>parameters that take pipeline input.<br>At line:1 char:59<br>+ python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content<br>+ ~~~~~~~~~~~<br> + CategoryInfo : InvalidArgument: (''':PSObject) [Get-Content], ParameterBindingExcepti<br> on<br> + FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand<br><br>Get-Content : The input object cannot be bound to any parameters for the command either because the<br>command does not take pipeline input or the input and its properties do not match any of the<br>parameters that take pipeline input.<br>At line:1 char:59<br>+ python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content<br>+ ~~~~~~~~~~~<br> + CategoryInfo : InvalidArgument: (x3d.py package ...h X3D Graphics!:PSObject) [Get-Con<br> tent], ParameterBindingException<br> + FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand<br><br>Get-Content : The input object cannot be bound to any parameters for the command either because the<br>command does not take pipeline input or the input and its properties do not match any of the<br>parameters that take pipeline input.<br>At line:1 char:59<br>+ python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content<br>+ ~~~~~~~~~~~<br> + CategoryInfo : InvalidArgument: (''':PSObject) [Get-Content], ParameterBindingExcepti<br> on<br> + FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand<br><br>Get-Content : The input object cannot be bound to any parameters for the command either because the<br>command does not take pipeline input or the input and its properties do not match any of the<br>parameters that take pipeline input.<br>At line:1 char:59<br>+ python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content<br>+ ~~~~~~~~~~~<br> + CategoryInfo : InvalidArgument: (from x3d import *:PSObject) [Get-Content], Parameter<br> BindingException<br> + FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand<br><br>Get-Content : The input object cannot be bound to any parameters for the command either because the<br>command does not take pipeline input or the input and its properties do not match any of the<br>parameters that take pipeline input.<br>At line:1 char:59<br>+ python .\x3djsonld.py ..\data\TextSpecialCharacters.json| Get-Content<br>+ ~~~~~~~~~~~<br> + CategoryInfo : InvalidArgument: (print(:PSObject) [Get-Content], ParameterBindingExce<br> ption<br> + FullyQualifiedErrorId : InputObjectNotBound,Microsoft.PowerShell.Commands.GetContentCommand<br></div></div>
</blockquote></div>
</blockquote></div>