[x3d-public] x3d.py pip packake problems and potential solution

Thu Apr 23 16:21:56 PDT 2020

ok, delving deeper into python packages versus modules, I think I
found the way to import x3d with the current PyPi package after
installation:

$ python
Python 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 22:33:48)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import x3d.x3d
x3d.py package loaded, have fun with X3D Graphics!
>>> help(x3d.x3d)

Note the repeated x3d.x3d. This refers to the package directory x3d
and then to the module file x3d.py . This works with or without the
unmodified __init__.py file.

It looks awkward but may be would you prefer. If it is documented I
think it would be ok since the import and from statements allow for
more convenient naming later.

-Andreas

On Thu, Apr 23, 2020 at 7:00 PM Andreas Plesch <andreasplesch at gmail.com> wrote:
>
> I tried to investigate this some more with x3d-0.0.27. I think one
> consequence of having the same name ('x3d') for the package and for
> the source file ('x3d.py') is that if you import x3d by referencing
> the source file, the compiled python gets cached and then possibly
> reused even after the source is removed and only the package install
> should be available.
>
> I would recommend considering renaming the package (say to x3dsai or
> X3D) or renaming x3d.py to say x3dclasses.py . Either way, I think,
> the 'from x3dclasses import *' line will be still necessary in
> __init__.py, for the package. Looking through various packages, this
> seems to be not an uncommon pattern for the __init__.py file. This
> kind of renaming will also make it easier to find problems with
> packaging.
>
> -Andreas
>
> On Thu, Apr 23, 2020 at 3:43 PM Don Brutzman <brutzman at nps.edu> wrote:
> >
> > Thanks for detailed explanation.  I'll read up further on this.
> >
> > Loren:  hope you can look at this issue with us also, seems fundamental.
> >
> >
> > On 4/23/2020 12:24 PM, Andreas Plesch wrote:
> > > Hi Don,
> > >
> > > PyPi requirements are met but that does not necessarily mean there is
> > > any functionality.
> > >
> > > I think you still need to add this line to the autogeneration of __init__.py
> > >
> > > # import the x3d.py module
> > > from x3d import *
> > >
> > > since I did not see it in the file you attached. Without it, the
> > > package is installed and can be imported but the imported object does
> > > not have any attributes or methods. With it, the x3d module from
> > > x3d.py gets imported and then the classes it provides exported as
> > > properties of the (new) module provided by the package.
> > >
> > > It is a bit confusing since in the __init__.py file the "x3d" in "from
> > > x3d import *" refers to the x3d.py file included in the package while
> > > after installation of the x3d PyPi package 'import x3d' in a python
> > > script refers to the imported package.
> > >
> > > I am not a PyPi expert and there may be other ways to properly package
> > > but adding the above line seemed like a good solution. I think without
> > > it the python system is not aware of the x3d.py file and the objects
> > > it provides.
> > >
> > > Can you reproduce the error below on a system which does not
> > > previously have x3d.py anywhere ?
> > >
> > >>> $ pip install x3d
> > >>> Collecting x3d
> > >>>     Downloading https://files.pythonhosted.org/packages/3b/4b/2a7cb8f738e5bf03beb729989fc1c0f52a86ddf61ea3fb38c61c55afd41b/x3d-0.0.26-py3-none-any.whl
> > >>> (204kB)
> > >>>        |████████████████████████████████| 204kB 5.6MB/s
> > >>> Installing collected packages: x3d
> > >>> Successfully installed x3d-0.0.26
> > >>> $ python
> > >>> Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
> > >>> [GCC 7.3.0] :: Anaconda, Inc. on linux
> > >>> Type "help", "copyright", "credits" or "license" for more information.
> > >>>>>> import x3d
> > >>>>>> x3d.X3D()
> > >>> Traceback (most recent call last):
> > >>>     File "<stdin>", line 1, in <module>
> > >>> AttributeError: module 'x3d' has no attribute 'X3D'
> > >
> > > -Andreas
> > >
> > > On Thu, Apr 23, 2020 at 2:46 PM Don Brutzman <brutzman at nps.edu> wrote:
> > >>
> > >> Thanks for your note Andreas.  __init__.py is autogenerated from X3DUOM whenever producing x3d.py package.
> > >>
> > >> Latest is attached, also online at
> > >>
> > >> * https://sourceforge.net/p/x3d/code/30232/tree/www.web3d.org/x3d/stylesheets/python/x3d/
> > >>
> > >> * https://sourceforge.net/p/x3d/code/30232/tree/www.web3d.org/x3d/stylesheets/python/x3d/__init__.py
> > >>
> > >> Just rechecked, yes 'X3D' is present there on line following # Statements
> > >>
> > >> Perhaps __init__.py isn't included properly?? Everything seems to pass PyPi requirements.
> > >>
> > >> I suspect you have to either use 'X3D' or 'x3d.X3D' according to how you have imported.  All the examples I'm testing/producing avoid the prefix, but I think there is a unit test in there.  Can add more tests, or improve documentation, as you think best.
> > >>
> > >> Shouldn't matter but am using latest Python (currently 3.8.2) in my testing.
> > >>
> > >> So, not seeing something to fix... let's persist please until this is sorted out satisfactorily.
> > >>
> > >>
> > >>
> > >> On 4/23/2020 7:00 AM, Andreas Plesch wrote:
> > >>> Importing the x3d.py module from the x3d.py file works (if the x3d.py
> > >>> file is in te python path).
> > >>>
> > >>> For convenience, there is also x3d python package (pip) which is
> > >>> available from the pip registry and can be installed by:
> > >>>
> > >>> pip install x3d
> > >>>
> > >>> to the system python modules.
> > >>>
> > >>> However, this does not quite work:
> > >>>
> > >>> $ pip install x3d
> > >>> Collecting x3d
> > >>>     Downloading https://files.pythonhosted.org/packages/3b/4b/2a7cb8f738e5bf03beb729989fc1c0f52a86ddf61ea3fb38c61c55afd41b/x3d-0.0.26-py3-none-any.whl
> > >>> (204kB)
> > >>>        |████████████████████████████████| 204kB 5.6MB/s
> > >>> Installing collected packages: x3d
> > >>> Successfully installed x3d-0.0.26
> > >>> jovyan at jupyter-gesiscss-2dnotebo-2dgetting-5fstarted-2d7wbaxlkp:~$ python
> > >>> Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
> > >>> [GCC 7.3.0] :: Anaconda, Inc. on linux
> > >>> Type "help", "copyright", "credits" or "license" for more information.
> > >>>>>> import x3d
> > >>>>>> x3d.X3D()
> > >>> Traceback (most recent call last):
> > >>>     File "<stdin>", line 1, in <module>
> > >>> AttributeError: module 'x3d' has no attribute 'X3D'
> > >>>
> > >>> I think what is needed is to actually import the module in the
> > >>> __init__.py file for the pip package.
> > >>>
> > >>> # __init__.py needed for properly configuring pypi distribution of
> > >>> x3d.py package
> > >>>
> > >>> # According to _Learning Python_ by Mark Lutz, fifth edition:
> > >>> # - Empty __init.py__ no longer required as of Python 3.3.  p. 761
> > >>> # - Using __init.py__ is performance advantage for loading, even when
> > >>> empty.  p. 761
> > >>> # - Using __all__ list to define exported values for import * is
> > >>> allowed but not required.  p. 735 and 771-772.
> > >>>
> > >>> # 6.4.1. Importing * From a Package
> > >>> # https://docs.python.org/3/tutorial/modules.html#importing-from-a-package
> > >>>
> > >>> # indicates that
> > >>> #     from packagename import *
> > >>> # "then imports whatever names are defined in the package" and
> > >>> # "Although certain modules are designed to export only names that
> > >>> follow certain patterns when you use import *,
> > >>> #     it is still considered bad practice in production code."
> > >>>
> > >>> # TODO testing continues to fix x3d.py package's class visibility
> > >>> satisfactorily for end users
> > >>>
> > >>> ## import the x3d.py module
> > >>> from x3d import *
> > >>>
> > >>> __all__ = [
> > >>>       # Field types
> > >>>
> > >>> With that change in the __init__.py file, the pip package works for me:
> > >>>
> > >>> $ python
> > >>> Python 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 22:33:48)
> > >>> [GCC 7.3.0] on linux
> > >>> Type "help", "copyright", "credits" or "license" for more information.
> > >>>>>> import x3d
> > >>> x3d.py package loaded, have fun with X3D Graphics!
> > >>>>>> x3d.X3D
> > >>> <class 'x3d.X3D'>
> > >>>>>> x3d.Box().toXML()
> > >>> '<Box/>\n'
> > >>>>>>
> > >>>
> > >>> Perhaps this change could be applied to the next release of the x3d
> > >>> pip package. I could not find the __init__.py file on sourceforge.
> > >>>
> > >>> Cheers,
> > >>>
> > >>> -Andreas
> > >>>
> > >>
> > >> all the best, Don
> > >> --
> > >> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> > >> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> > >> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
> > >
> > >
> > >
> >
> > all the best, Don
> > --
> > Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> > X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
>
>
>
> --
> Andreas Plesch
> Waltham, MA 02453

-- 
Andreas Plesch
Waltham, MA 02453