[x3d-public] x3d.py pip packake problems and potential solution

Thu Apr 23 16:00:31 PDT 2020

I tried to investigate this some more with x3d-0.0.27. I think one
consequence of having the same name ('x3d') for the package and for
the source file ('x3d.py') is that if you import x3d by referencing
the source file, the compiled python gets cached and then possibly
reused even after the source is removed and only the package install
should be available.

I would recommend considering renaming the package (say to x3dsai or
X3D) or renaming x3d.py to say x3dclasses.py . Either way, I think,
the 'from x3dclasses import *' line will be still necessary in
__init__.py, for the package. Looking through various packages, this
seems to be not an uncommon pattern for the __init__.py file. This
kind of renaming will also make it easier to find problems with
packaging.

-Andreas

On Thu, Apr 23, 2020 at 3:43 PM Don Brutzman <brutzman at nps.edu> wrote:
>
> Thanks for detailed explanation.  I'll read up further on this.
>
> Loren:  hope you can look at this issue with us also, seems fundamental.
>
>
> On 4/23/2020 12:24 PM, Andreas Plesch wrote:
> > Hi Don,
> >
> > PyPi requirements are met but that does not necessarily mean there is
> > any functionality.
> >
> > I think you still need to add this line to the autogeneration of __init__.py
> >
> > # import the x3d.py module
> > from x3d import *
> >
> > since I did not see it in the file you attached. Without it, the
> > package is installed and can be imported but the imported object does
> > not have any attributes or methods. With it, the x3d module from
> > x3d.py gets imported and then the classes it provides exported as
> > properties of the (new) module provided by the package.
> >
> > It is a bit confusing since in the __init__.py file the "x3d" in "from
> > x3d import *" refers to the x3d.py file included in the package while
> > after installation of the x3d PyPi package 'import x3d' in a python
> > script refers to the imported package.
> >
> > I am not a PyPi expert and there may be other ways to properly package
> > but adding the above line seemed like a good solution. I think without
> > it the python system is not aware of the x3d.py file and the objects
> > it provides.
> >
> > Can you reproduce the error below on a system which does not
> > previously have x3d.py anywhere ?
> >
> >>> $ pip install x3d
> >>> Collecting x3d
> >>>     Downloading https://files.pythonhosted.org/packages/3b/4b/2a7cb8f738e5bf03beb729989fc1c0f52a86ddf61ea3fb38c61c55afd41b/x3d-0.0.26-py3-none-any.whl
> >>> (204kB)
> >>>        |████████████████████████████████| 204kB 5.6MB/s
> >>> Installing collected packages: x3d
> >>> Successfully installed x3d-0.0.26
> >>> $ python
> >>> Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
> >>> [GCC 7.3.0] :: Anaconda, Inc. on linux
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >>>>>> import x3d
> >>>>>> x3d.X3D()
> >>> Traceback (most recent call last):
> >>>     File "<stdin>", line 1, in <module>
> >>> AttributeError: module 'x3d' has no attribute 'X3D'
> >
> > -Andreas
> >
> > On Thu, Apr 23, 2020 at 2:46 PM Don Brutzman <brutzman at nps.edu> wrote:
> >>
> >> Thanks for your note Andreas.  __init__.py is autogenerated from X3DUOM whenever producing x3d.py package.
> >>
> >> Latest is attached, also online at
> >>
> >> * https://sourceforge.net/p/x3d/code/30232/tree/www.web3d.org/x3d/stylesheets/python/x3d/
> >>
> >> * https://sourceforge.net/p/x3d/code/30232/tree/www.web3d.org/x3d/stylesheets/python/x3d/__init__.py
> >>
> >> Just rechecked, yes 'X3D' is present there on line following # Statements
> >>
> >> Perhaps __init__.py isn't included properly?? Everything seems to pass PyPi requirements.
> >>
> >> I suspect you have to either use 'X3D' or 'x3d.X3D' according to how you have imported.  All the examples I'm testing/producing avoid the prefix, but I think there is a unit test in there.  Can add more tests, or improve documentation, as you think best.
> >>
> >> Shouldn't matter but am using latest Python (currently 3.8.2) in my testing.
> >>
> >> So, not seeing something to fix... let's persist please until this is sorted out satisfactorily.
> >>
> >>
> >>
> >> On 4/23/2020 7:00 AM, Andreas Plesch wrote:
> >>> Importing the x3d.py module from the x3d.py file works (if the x3d.py
> >>> file is in te python path).
> >>>
> >>> For convenience, there is also x3d python package (pip) which is
> >>> available from the pip registry and can be installed by:
> >>>
> >>> pip install x3d
> >>>
> >>> to the system python modules.
> >>>
> >>> However, this does not quite work:
> >>>
> >>> $ pip install x3d
> >>> Collecting x3d
> >>>     Downloading https://files.pythonhosted.org/packages/3b/4b/2a7cb8f738e5bf03beb729989fc1c0f52a86ddf61ea3fb38c61c55afd41b/x3d-0.0.26-py3-none-any.whl
> >>> (204kB)
> >>>        |████████████████████████████████| 204kB 5.6MB/s
> >>> Installing collected packages: x3d
> >>> Successfully installed x3d-0.0.26
> >>> jovyan at jupyter-gesiscss-2dnotebo-2dgetting-5fstarted-2d7wbaxlkp:~$ python
> >>> Python 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
> >>> [GCC 7.3.0] :: Anaconda, Inc. on linux
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >>>>>> import x3d
> >>>>>> x3d.X3D()
> >>> Traceback (most recent call last):
> >>>     File "<stdin>", line 1, in <module>
> >>> AttributeError: module 'x3d' has no attribute 'X3D'
> >>>
> >>> I think what is needed is to actually import the module in the
> >>> __init__.py file for the pip package.
> >>>
> >>> # __init__.py needed for properly configuring pypi distribution of
> >>> x3d.py package
> >>>
> >>> # According to _Learning Python_ by Mark Lutz, fifth edition:
> >>> # - Empty __init.py__ no longer required as of Python 3.3.  p. 761
> >>> # - Using __init.py__ is performance advantage for loading, even when
> >>> empty.  p. 761
> >>> # - Using __all__ list to define exported values for import * is
> >>> allowed but not required.  p. 735 and 771-772.
> >>>
> >>> # 6.4.1. Importing * From a Package
> >>> # https://docs.python.org/3/tutorial/modules.html#importing-from-a-package
> >>>
> >>> # indicates that
> >>> #     from packagename import *
> >>> # "then imports whatever names are defined in the package" and
> >>> # "Although certain modules are designed to export only names that
> >>> follow certain patterns when you use import *,
> >>> #     it is still considered bad practice in production code."
> >>>
> >>> # TODO testing continues to fix x3d.py package's class visibility
> >>> satisfactorily for end users
> >>>
> >>> ## import the x3d.py module
> >>> from x3d import *
> >>>
> >>> __all__ = [
> >>>       # Field types
> >>>
> >>> With that change in the __init__.py file, the pip package works for me:
> >>>
> >>> $ python
> >>> Python 3.7.6 | packaged by conda-forge | (default, Jan  7 2020, 22:33:48)
> >>> [GCC 7.3.0] on linux
> >>> Type "help", "copyright", "credits" or "license" for more information.
> >>>>>> import x3d
> >>> x3d.py package loaded, have fun with X3D Graphics!
> >>>>>> x3d.X3D
> >>> <class 'x3d.X3D'>
> >>>>>> x3d.Box().toXML()
> >>> '<Box/>\n'
> >>>>>>
> >>>
> >>> Perhaps this change could be applied to the next release of the x3d
> >>> pip package. I could not find the __init__.py file on sourceforge.
> >>>
> >>> Cheers,
> >>>
> >>> -Andreas
> >>>
> >>
> >> all the best, Don
> >> --
> >> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> >> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> >> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
> >
> >
> >
>
> all the best, Don
> --
> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman

-- 
Andreas Plesch
Waltham, MA 02453