Re: Problems uploading jars to update site

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Problems uploading jars to update site

Curtis Rueden
Hi Birgit,

[Switching to imagej-devel, to avoid spamming ImageJ users with technical mumbo jumbo.]

>  the Xalan jar is included in our dependencies only to due loci_tools
>  declaring that jar as a dependency. As during the update procedure we
>  were also told that loci_tools is now part of the bioformats package,
>  I wonder if we can get rid of that dependency by switching from
>  loci_tools to bioformats_package.

From a Maven perspective, I would suggest depending on the actual Bio-Formats components you are using, rather than any uber-JAR such as loci_tools (deprecated) or bioformats_package (the current uber-jar).

There is a brief discussion of the pros and cons of uber-jars from an ImageJ perspective here:

If you point me at the public SCM for your project, I'll gladly take a look at your POM and make some suggestions. With Bio-Formats, the standard approach is to add compile-time dependency on ome:formats-api:5.0.8, and runtime dependencies on ome:formats-bsd:5.0.8 and ome:formats-gpl:5.0.8, since those latter two components provide file format implementations ("plugin"-esque) which are not needed at compile time, but you want on the classpath at runtime for all the file format support.

Regards,
Curtis

On Fri, Mar 20, 2015 at 7:57 AM, Birgit Möller <[hidden email]> wrote:
Hi Curtis!

On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden <[hidden email]> wrote:
>Hi Birgit,
>
>> we are trying to set up an ImageJ update site for our plugin
>> collection MiToBo.
>
>Sounds great!
>
>> Since we have a lot of dependencies we would like to test the setup
>> first on our own local server
>
>Good idea.
>
>> but in the end we plan to provide Mitobo via an ImageJ Wiki update
>> site.
>
>Well, if you have a dedicated server, you can just use that. There is no
>requirement to use a personal update site. But I guess the URL "
>http://sites.imagej.net/MiToBo" would be pretty friendly. :-)

yes, I agree, and we will go for it soon :-)

>> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> jars seem to have been provided by the Fiji Update Site in former
>> days, but are now declared obsolete.
>
>Indeed.
>
>> Since we need both of them we tried to upload them to our own site,
>> however, this fails. First Fiji claims about changed checksums for
>> both jar files. After recalculating the checksums and trying to
>> upload them again, the updater throws the following exception:
>>
>> Upload failed: java.lang.NoClassDefFoundError:
>> org/apache/xml/serializer/TreeWalker
>
>How bizarre. I wonder if this is something xalan-specific, since xalan
>ships classes that are also part of the JRE itself. As a test, you could
>temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
>the sezpoz JAR to your update site, and see whether you encounter the same
>issue.

It looks like it is really a Xalan issue. Skipping the Xalan jar everything works fine. Meanwhile I also figured out that the Xalan jar is included in our dependencies only to due loci_tools declaring that jar as a dependency. As during the update procedure we were also told that loci_tools is now part of the bioformats package, I wonder if we can get rid of that dependency by switching from loci_tools to bioformats_package. Moreover, if the Xalan jar is not present, our operators and plugins seem to work anyway, so maybe we even do not really need any stuff contained in that jar - but we will still need to investigate that further.

>[...]
>
>Regards,
>Curtis
>
>[...]

For now, thanks for your elaborate reply.  If we encounter additional problems, I will get back to you and the list again. But, as suggested, next time I will post questions regarding update sites on the imagej-devel list instead of this one.
Thanks and kind regards,

 Birgit

---

On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden <[hidden email]> wrote:

>Hi Birgit,
>
>> we are trying to set up an ImageJ update site for our plugin
>> collection MiToBo.
>
>Sounds great!
>
>> Since we have a lot of dependencies we would like to test the setup
>> first on our own local server
>
>Good idea.
>
>> but in the end we plan to provide Mitobo via an ImageJ Wiki update
>> site.
>
>Well, if you have a dedicated server, you can just use that. There is no
>requirement to use a personal update site. But I guess the URL "
>http://sites.imagej.net/MiToBo" would be pretty friendly. :-)
>
>> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> jars seem to have been provided by the Fiji Update Site in former
>> days, but are now declared obsolete.
>
>Indeed.
>
>> Since we need both of them we tried to upload them to our own site,
>> however, this fails. First Fiji claims about changed checksums for
>> both jar files. After recalculating the checksums and trying to
>> upload them again, the updater throws the following exception:
>>
>> Upload failed: java.lang.NoClassDefFoundError:
>> org/apache/xml/serializer/TreeWalker
>
>How bizarre. I wonder if this is something xalan-specific, since xalan
>ships classes that are also part of the JRE itself. As a test, you could
>temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
>the sezpoz JAR to your update site, and see whether you encounter the same
>issue.
>
>> What is Fiji's strategy to deal with cases where the same jar in
>> different versions is provided by two update sites?
>
>IIRC, the ImageJ Updater (which is part of ImageJ2, and not specific to the
>Fiji distribution) favors versions further down in the list of update
>sites. That is: there is a linear order to the update sites, such that
>files from sites further down the chain are considered to "shadow" the same
>file from sites further up the chain. The reason I say "IIRC" is because I
>am not 100% certain that chain order is only defined by the ordering listed
>on the "List of update sites" wiki page. It may be that the order changes
>for a local installation depending on when sites are toggled on and off. If
>you are curious to dig in further, there is a unit test that verifies that
>various multi-update-site features work properly [1], which you could play
>with.
>
>The gist is that there is currently _no_ mechanism for defining
>update-site-level dependencies, beyond just building on top of the ImageJ
>and Fiji update sites. It is currently the user's responsibility to enable
>update sites upon which your update site depends. Some day we may address
>that issue as the number of update sites continues to grow, but it is not a
>simple feature to add.
>
>> How does the updater try to determine depencies of plugins? In our
>> case while checking our local jars it claimed to detect a cyclic
>> dependency which was not there.
>
>It uses byte-code analysis to detect the dependencies. However, you can
>tweak them before uploading by editing the list of dependencies directly in
>the right-hand text box of the graphical updater window. This is sometimes
>necessary in cases where there are undesired circular dependencies (e.g.,
>slf4j-api and various slf4j bindings always detect as circular, due to the
>design of SLF4J).
>
>Regards,
>Curtis
>
>[1]
>https://github.com/imagej/imagej-updater/blob/imagej-updater-0.7.1/src/test/java/net/imagej/updater/MultipleSitesTest.java
>
>On Sun, Mar 15, 2015 at 7:34 AM, Birgit Möller <
>[hidden email]> wrote:
>
>> Dear all,
>> we are trying to set up an ImageJ update site for our plugin collection
>> MiToBo. Since we have a lot of dependencies we would like to test the setup
>> first on our own local server, but in the end we plan to provide Mitobo via
>> an ImageJ Wiki update site. Unfortunately we encountered some problems.
>> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both jars
>> seem to have been provided by the Fiji Update Site in former days, but are
>> now declared obsolete. Since we need both of them we tried to upload them
>> to our own site, however, this fails. First Fiji claims about changed
>> checksums for both jar files. After recalculating the checksums and trying
>> to upload them again, the updater throws the following exception:
>>
>> Upload failed: java.lang.NoClassDefFoundError:
>> org/apache/xml/serializer/TreeWalker
>>
>> Then the updater can only be canceled and trying to re-run it requires
>> recalculation of checksums again. Nevertheless the upload fails again with
>> the above exception. Does anyone have an idea why it is not possible to
>> upload both jars to our own site? For other jars unseen by Fiji before
>> there were no problems.
>>
>> And finally two general questions:
>>
>> - What is Fiji's strategy to deal with cases where the same jar in
>> different versions is provided by two update sites?
>>
>> - How does the updater try to determine depencies of plugins? In our case
>> while checking our local jars it claimed to detect a cyclic dependency
>> which was not there.
>>
>> Thanks and best regards,
>>
>>  Birgit
>>
>>
>> ------------------------------------------------------------------------
>> Dr. rer. nat. Birgit Moeller
>>
>> Pattern Recognition & Bioinformatics
>> Institute of Computer Science / Faculty of Natural Sciences III
>> Martin Luther University Halle-Wittenberg
>>
>> office:     Room 4.12
>> phone:      <a href="tel:%2B49%280%29345-55-24745" value="+493455524745">+49(0)345-55-24745
>> fax:        <a href="tel:%2B49%280%29345-55-27039" value="+493455527039">+49(0)345-55-27039
>> snail mail: Von-Seckendorff-Platz 1, 06120 Halle / Saale (Germany)
>> e-mail:     [hidden email]
>> www:        http://www.informatik.uni-halle.de/moeller/
>> ------------------------------------------------------------------------
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
>--
>ImageJ mailing list: http://imagej.nih.gov/ij/list.html

--
ImageJ mailing list: http://imagej.nih.gov/ij/list.html


_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problems uploading jars to update site

Birgit Moeller
Hi Curtis!

Curtis Rueden wrote
Hi Birgit,
[...]
From a Maven perspective, I would suggest depending on the actual
Bio-Formats components you are using, rather than any uber-JAR such as
loci_tools (deprecated) or bioformats_package (the current uber-jar).

There is a brief discussion of the pros and cons of uber-jars from an
ImageJ perspective here:

http://imagej.net/Frequently_Asked_Questions#How_can_I_call_ImageJ_from_my_software.3F
I totally agree that depending on the actual jars containing the components has many advantages. Using the uber-jars in Alida/Mitobo is mainly a relict of former days. In the beginning it was the easiest to simply use the uber-jars to not miss anything. And until now we did not think much about updating. But, for sure it would be best to change that now and we will do that.

Curtis Rueden wrote
If you point me at the public SCM for your project, I'll gladly take a look
at your POM and make some suggestions. With Bio-Formats, the standard
approach is to add compile-time dependency on ome:formats-api:5.0.8, and
runtime dependencies on ome:formats-bsd:5.0.8 and ome:formats-gpl:5.0.8,
since those latter two components provide file format implementations
("plugin"-esque) which are not needed at compile time, but you want on the
classpath at runtime for all the file format support.

Regards,
Curtis
We currently don't have an SCM being publically accessible, and our Maven server crashed some days ago, so unfortunately I cannot point you to the deployed artifacts or source directly. I will let you know immediately, when our server is up again. But, in the meantime I can already tell you on what functionality we currently depend being related to ImageJ and Bioformats. Probably you can already give me a hint based on this information. We define the following three dependencies:

    <dependency>
      <groupId>loci</groupId>
      <artifactId>loci_tools</artifactId>
      <version>4.4.8</version>
    </dependency>
    <dependency>
      <groupId>net.imglib2</groupId>
      <artifactId>imglib2</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>net.imglib2</groupId>
      <artifactId>imglib2-algorithms</artifactId>
      <version>2.0.0-beta-10</version>
    </dependency>

We basically use functionality from the following Bioformats/Loci packages:
import loci.common.*;
import loci.common.services.*;
import loci.formats.*;
import loci.formats.gui.*;
import loci.formats.meta.*;
import loci.formats.out.*;
import loci.formats.services.*;
import ome.xml.model.*;

With regard to Imglib2 we use basic functionality from

net.imglib2.img
net.imglib2.type

as well as component trees to be found in

net.imglib2.algorithm.componenttree.

Regards,

 Birgit



Curtis Rueden wrote
On Fri, Mar 20, 2015 at 7:57 AM, Birgit Möller <
[hidden email]> wrote:

> Hi Curtis!
>
> On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden <[hidden email]>
> wrote:
> >Hi Birgit,
> >
> >> we are trying to set up an ImageJ update site for our plugin
> >> collection MiToBo.
> >
> >Sounds great!
> >
> >> Since we have a lot of dependencies we would like to test the setup
> >> first on our own local server
> >
> >Good idea.
> >
> >> but in the end we plan to provide Mitobo via an ImageJ Wiki update
> >> site.
> >
> >Well, if you have a dedicated server, you can just use that. There is no
> >requirement to use a personal update site. But I guess the URL "
> >http://sites.imagej.net/MiToBo" would be pretty friendly. :-)
>
> yes, I agree, and we will go for it soon :-)
>
> >> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
> >> jars seem to have been provided by the Fiji Update Site in former
> >> days, but are now declared obsolete.
> >
> >Indeed.
> >
> >> Since we need both of them we tried to upload them to our own site,
> >> however, this fails. First Fiji claims about changed checksums for
> >> both jar files. After recalculating the checksums and trying to
> >> upload them again, the updater throws the following exception:
> >>
> >> Upload failed: java.lang.NoClassDefFoundError:
> >> org/apache/xml/serializer/TreeWalker
> >
> >How bizarre. I wonder if this is something xalan-specific, since xalan
> >ships classes that are also part of the JRE itself. As a test, you could
> >temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
> >the sezpoz JAR to your update site, and see whether you encounter the same
> >issue.
>
> It looks like it is really a Xalan issue. Skipping the Xalan jar
> everything works fine. Meanwhile I also figured out that the Xalan jar is
> included in our dependencies only to due loci_tools declaring that jar as a
> dependency. As during the update procedure we were also told that
> loci_tools is now part of the bioformats package, I wonder if we can get
> rid of that dependency by switching from loci_tools to bioformats_package.
> Moreover, if the Xalan jar is not present, our operators and plugins seem
> to work anyway, so maybe we even do not really need any stuff contained in
> that jar - but we will still need to investigate that further.
>
> >[...]
> >
> >Regards,
> >Curtis
> >
> >[...]
>
> For now, thanks for your elaborate reply.  If we encounter additional
> problems, I will get back to you and the list again. But, as suggested,
> next time I will post questions regarding update sites on the imagej-devel
> list instead of this one.
> Thanks and kind regards,
>
>  Birgit
>
> ---
>
> On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden <[hidden email]>
> wrote:
>
> >Hi Birgit,
> >
> >> we are trying to set up an ImageJ update site for our plugin
> >> collection MiToBo.
> >
> >Sounds great!
> >
> >> Since we have a lot of dependencies we would like to test the setup
> >> first on our own local server
> >
> >Good idea.
> >
> >> but in the end we plan to provide Mitobo via an ImageJ Wiki update
> >> site.
> >
> >Well, if you have a dedicated server, you can just use that. There is no
> >requirement to use a personal update site. But I guess the URL "
> >http://sites.imagej.net/MiToBo" would be pretty friendly. :-)
> >
> >> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
> >> jars seem to have been provided by the Fiji Update Site in former
> >> days, but are now declared obsolete.
> >
> >Indeed.
> >
> >> Since we need both of them we tried to upload them to our own site,
> >> however, this fails. First Fiji claims about changed checksums for
> >> both jar files. After recalculating the checksums and trying to
> >> upload them again, the updater throws the following exception:
> >>
> >> Upload failed: java.lang.NoClassDefFoundError:
> >> org/apache/xml/serializer/TreeWalker
> >
> >How bizarre. I wonder if this is something xalan-specific, since xalan
> >ships classes that are also part of the JRE itself. As a test, you could
> >temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
> >the sezpoz JAR to your update site, and see whether you encounter the same
> >issue.
> >
> >> What is Fiji's strategy to deal with cases where the same jar in
> >> different versions is provided by two update sites?
> >
> >IIRC, the ImageJ Updater (which is part of ImageJ2, and not specific to
> the
> >Fiji distribution) favors versions further down in the list of update
> >sites. That is: there is a linear order to the update sites, such that
> >files from sites further down the chain are considered to "shadow" the
> same
> >file from sites further up the chain. The reason I say "IIRC" is because I
> >am not 100% certain that chain order is only defined by the ordering
> listed
> >on the "List of update sites" wiki page. It may be that the order changes
> >for a local installation depending on when sites are toggled on and off.
> If
> >you are curious to dig in further, there is a unit test that verifies that
> >various multi-update-site features work properly [1], which you could play
> >with.
> >
> >The gist is that there is currently _no_ mechanism for defining
> >update-site-level dependencies, beyond just building on top of the ImageJ
> >and Fiji update sites. It is currently the user's responsibility to enable
> >update sites upon which your update site depends. Some day we may address
> >that issue as the number of update sites continues to grow, but it is not
> a
> >simple feature to add.
> >
> >> How does the updater try to determine depencies of plugins? In our
> >> case while checking our local jars it claimed to detect a cyclic
> >> dependency which was not there.
> >
> >It uses byte-code analysis to detect the dependencies. However, you can
> >tweak them before uploading by editing the list of dependencies directly
> in
> >the right-hand text box of the graphical updater window. This is sometimes
> >necessary in cases where there are undesired circular dependencies (e.g.,
> >slf4j-api and various slf4j bindings always detect as circular, due to the
> >design of SLF4J).
> >
> >Regards,
> >Curtis
> >
> >[1]
> >
> https://github.com/imagej/imagej-updater/blob/imagej-updater-0.7.1/src/test/java/net/imagej/updater/MultipleSitesTest.java
> >
> >On Sun, Mar 15, 2015 at 7:34 AM, Birgit Möller <
> >[hidden email]> wrote:
> >
> >> Dear all,
> >> we are trying to set up an ImageJ update site for our plugin collection
> >> MiToBo. Since we have a lot of dependencies we would like to test the
> setup
> >> first on our own local server, but in the end we plan to provide Mitobo
> via
> >> an ImageJ Wiki update site. Unfortunately we encountered some problems.
> >> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
> jars
> >> seem to have been provided by the Fiji Update Site in former days, but
> are
> >> now declared obsolete. Since we need both of them we tried to upload
> them
> >> to our own site, however, this fails. First Fiji claims about changed
> >> checksums for both jar files. After recalculating the checksums and
> trying
> >> to upload them again, the updater throws the following exception:
> >>
> >> Upload failed: java.lang.NoClassDefFoundError:
> >> org/apache/xml/serializer/TreeWalker
> >>
> >> Then the updater can only be canceled and trying to re-run it requires
> >> recalculation of checksums again. Nevertheless the upload fails again
> with
> >> the above exception. Does anyone have an idea why it is not possible to
> >> upload both jars to our own site? For other jars unseen by Fiji before
> >> there were no problems.
> >>
> >> And finally two general questions:
> >>
> >> - What is Fiji's strategy to deal with cases where the same jar in
> >> different versions is provided by two update sites?
> >>
> >> - How does the updater try to determine depencies of plugins? In our
> case
> >> while checking our local jars it claimed to detect a cyclic dependency
> >> which was not there.
> >>
> >> Thanks and best regards,
> >>
> >>  Birgit
> >>
> >>
> >> ------------------------------------------------------------------------
> >> Dr. rer. nat. Birgit Moeller
> >>
> >> Pattern Recognition & Bioinformatics
> >> Institute of Computer Science / Faculty of Natural Sciences III
> >> Martin Luther University Halle-Wittenberg
> >>
> >> office:     Room 4.12
> >> phone:      +49(0)345-55-24745
> >> fax:        +49(0)345-55-27039
> >> snail mail: Von-Seckendorff-Platz 1, 06120 Halle / Saale (Germany)
> >> e-mail:     [hidden email]
> >> www:        http://www.informatik.uni-halle.de/moeller/
> >> ------------------------------------------------------------------------
> >>
> >> --
> >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
> >>
> >
> >--
> >ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>
> --
> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>

_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problems uploading jars to update site

Curtis Rueden
Hi Birgit,

So your main deps are:

> loci:loci_tools:4.4.8
> net.imglib2:imglib2:2.2.0
> net.imglib2:imglib2-algorithms:2.0.0-beta-10

There are a couple of problems.

1) The imglib2-algorithms artifact is obsolete. Since the 2.0.0 final release of imglib2, the Algorithms library was renamed to imglib2-algorithm instead, for consistency.

2) Bio-Formats 4.4.x is no longer shipped with Fiji. It currently ships Bio-Formats 5.0.x, which will soon become 5.1.0 once the OME team finalizes that version. MiToBo will have compatibility problems with Fiji as long as it still depends on 4.4.8.

I advise extending pom-imagej (or alternately, import it in your dependencyManagement section [1]) to inherit its Bill of Materials [2], to get consistency of versioning. Otherwise, you are just taking a stab in the dark at getting all the version numbers right. The BOM exists as a promise to downstream projects that all the versions declared therein will work together. In general, if you have not already read over the ImageJ Architecture page [3], I would suggest doing so.

As of this writing, if you extend or import the newest pom-imagej 6.0.0, you can then add the following dependencies:

> ome:formats-api (for loci.formats.*)
> ome:formats-common (for loci.common.*)
> ome:ome-xml (for ome.xml.*)
> ome:formats-bsd (runtime; for open file formats)
> ome:formats-gpl (runtime; for proprietary file formats)
> net.imglib2:imglib2
> net.imglib2:imglib2-algorithm

I did not bother to specify the version numbers above -- the whole point is that you would leave off the "<version>...</version>" lines in your POM, since they are managed by pom-imagej instead.

Happy to help if you need further information or assistance with this!

We currently don't have an SCM being publically accessible

If you are willing to change that, we would be delighted to add more MiToBo links in the relevant places of our SciJava propaganda. In particular:


And on the Architecture page too, adding (e.g.) a MiToBo GitHub organization, pom-mitobo, etc., to the various tables.

Currently we do include the MiToBo and Alida logos in our talks when we mention SciJava module interoperability, but it would be nice to link to more of your project resources.

Regards,
Curtis


On Mon, Mar 23, 2015 at 4:45 AM, Birgit Moeller <[hidden email]> wrote:
Hi Curtis!


Curtis Rueden wrote
> Hi Birgit,
> [...]
> From a Maven perspective, I would suggest depending on the actual
> Bio-Formats components you are using, rather than any uber-JAR such as
> loci_tools (deprecated) or bioformats_package (the current uber-jar).
>
> There is a brief discussion of the pros and cons of uber-jars from an
> ImageJ perspective here:
>
> http://imagej.net/Frequently_Asked_Questions#How_can_I_call_ImageJ_from_my_software.3F

I totally agree that depending on the actual jars containing the components
has many advantages. Using the uber-jars in Alida/Mitobo is mainly a relict
of former days. In the beginning it was the easiest to simply use the
uber-jars to not miss anything. And until now we did not think much about
updating. But, for sure it would be best to change that now and we will do
that.


Curtis Rueden wrote
> If you point me at the public SCM for your project, I'll gladly take a
> look
> at your POM and make some suggestions. With Bio-Formats, the standard
> approach is to add compile-time dependency on ome:formats-api:5.0.8, and
> runtime dependencies on ome:formats-bsd:5.0.8 and ome:formats-gpl:5.0.8,
> since those latter two components provide file format implementations
> ("plugin"-esque) which are not needed at compile time, but you want on the
> classpath at runtime for all the file format support.
>
> Regards,
> Curtis

We currently don't have an SCM being publically accessible, and our Maven
server crashed some days ago, so unfortunately I cannot point you to the
deployed artifacts or source directly. I will let you know immediately, when
our server is up again. But, in the meantime I can already tell you on what
functionality we currently depend being related to ImageJ and Bioformats.
Probably you can already give me a hint based on this information. We define
the following three dependencies:

    <dependency>
      <groupId>loci</groupId>
      <artifactId>loci_tools</artifactId>
      <version>4.4.8</version>
    </dependency>
    <dependency>
      <groupId>net.imglib2</groupId>
      <artifactId>imglib2</artifactId>
      <version>2.2.0</version>
    </dependency>
    <dependency>
      <groupId>net.imglib2</groupId>
      <artifactId>imglib2-algorithms</artifactId>
      <version>2.0.0-beta-10</version>
    </dependency>

We basically use functionality from the following Bioformats/Loci packages:
import loci.common.*;
import loci.common.services.*;
import loci.formats.*;
import loci.formats.gui.*;
import loci.formats.meta.*;
import loci.formats.out.*;
import loci.formats.services.*;
import ome.xml.model.*;

With regard to Imglib2 we use basic functionality from

net.imglib2.img
net.imglib2.type

as well as component trees to be found in

net.imglib2.algorithm.componenttree.

Regards,

 Birgit




Curtis Rueden wrote
> On Fri, Mar 20, 2015 at 7:57 AM, Birgit Möller <

> birgit.moeller@.uni-halle

>> wrote:
>
>> Hi Curtis!
>>
>> On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden &lt;

> ctrueden@

> &gt;
>> wrote:
>> >Hi Birgit,
>> >
>> >> we are trying to set up an ImageJ update site for our plugin
>> >> collection MiToBo.
>> >
>> >Sounds great!
>> >
>> >> Since we have a lot of dependencies we would like to test the setup
>> >> first on our own local server
>> >
>> >Good idea.
>> >
>> >> but in the end we plan to provide Mitobo via an ImageJ Wiki update
>> >> site.
>> >
>> >Well, if you have a dedicated server, you can just use that. There is no
>> >requirement to use a personal update site. But I guess the URL "
>> >http://sites.imagej.net/MiToBo" would be pretty friendly. :-)
>>
>> yes, I agree, and we will go for it soon :-)
>>
>> >> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> >> jars seem to have been provided by the Fiji Update Site in former
>> >> days, but are now declared obsolete.
>> >
>> >Indeed.
>> >
>> >> Since we need both of them we tried to upload them to our own site,
>> >> however, this fails. First Fiji claims about changed checksums for
>> >> both jar files. After recalculating the checksums and trying to
>> >> upload them again, the updater throws the following exception:
>> >>
>> >> Upload failed: java.lang.NoClassDefFoundError:
>> >> org/apache/xml/serializer/TreeWalker
>> >
>> >How bizarre. I wonder if this is something xalan-specific, since xalan
>> >ships classes that are also part of the JRE itself. As a test, you could
>> >temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
>> >the sezpoz JAR to your update site, and see whether you encounter the
>> same
>> >issue.
>>
>> It looks like it is really a Xalan issue. Skipping the Xalan jar
>> everything works fine. Meanwhile I also figured out that the Xalan jar is
>> included in our dependencies only to due loci_tools declaring that jar as
>> a
>> dependency. As during the update procedure we were also told that
>> loci_tools is now part of the bioformats package, I wonder if we can get
>> rid of that dependency by switching from loci_tools to
>> bioformats_package.
>> Moreover, if the Xalan jar is not present, our operators and plugins seem
>> to work anyway, so maybe we even do not really need any stuff contained
>> in
>> that jar - but we will still need to investigate that further.
>>
>> >[...]
>> >
>> >Regards,
>> >Curtis
>> >
>> >[...]
>>
>> For now, thanks for your elaborate reply.  If we encounter additional
>> problems, I will get back to you and the list again. But, as suggested,
>> next time I will post questions regarding update sites on the
>> imagej-devel
>> list instead of this one.
>> Thanks and kind regards,
>>
>>  Birgit
>>
>> ---
>>
>> On Thu, 19 Mar 2015 12:37:30 -0500, Curtis Rueden &lt;

> ctrueden@

> &gt;
>> wrote:
>>
>> >Hi Birgit,
>> >
>> >> we are trying to set up an ImageJ update site for our plugin
>> >> collection MiToBo.
>> >
>> >Sounds great!
>> >
>> >> Since we have a lot of dependencies we would like to test the setup
>> >> first on our own local server
>> >
>> >Good idea.
>> >
>> >> but in the end we plan to provide Mitobo via an ImageJ Wiki update
>> >> site.
>> >
>> >Well, if you have a dedicated server, you can just use that. There is no
>> >requirement to use a personal update site. But I guess the URL "
>> >http://sites.imagej.net/MiToBo" would be pretty friendly. :-)
>> >
>> >> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> >> jars seem to have been provided by the Fiji Update Site in former
>> >> days, but are now declared obsolete.
>> >
>> >Indeed.
>> >
>> >> Since we need both of them we tried to upload them to our own site,
>> >> however, this fails. First Fiji claims about changed checksums for
>> >> both jar files. After recalculating the checksums and trying to
>> >> upload them again, the updater throws the following exception:
>> >>
>> >> Upload failed: java.lang.NoClassDefFoundError:
>> >> org/apache/xml/serializer/TreeWalker
>> >
>> >How bizarre. I wonder if this is something xalan-specific, since xalan
>> >ships classes that are also part of the JRE itself. As a test, you could
>> >temporarily delete your xalan JAR, restart ImageJ, and try to add _only_
>> >the sezpoz JAR to your update site, and see whether you encounter the
>> same
>> >issue.
>> >
>> >> What is Fiji's strategy to deal with cases where the same jar in
>> >> different versions is provided by two update sites?
>> >
>> >IIRC, the ImageJ Updater (which is part of ImageJ2, and not specific to
>> the
>> >Fiji distribution) favors versions further down in the list of update
>> >sites. That is: there is a linear order to the update sites, such that
>> >files from sites further down the chain are considered to "shadow" the
>> same
>> >file from sites further up the chain. The reason I say "IIRC" is because
>> I
>> >am not 100% certain that chain order is only defined by the ordering
>> listed
>> >on the "List of update sites" wiki page. It may be that the order
>> changes
>> >for a local installation depending on when sites are toggled on and off.
>> If
>> >you are curious to dig in further, there is a unit test that verifies
>> that
>> >various multi-update-site features work properly [1], which you could
>> play
>> >with.
>> >
>> >The gist is that there is currently _no_ mechanism for defining
>> >update-site-level dependencies, beyond just building on top of the
>> ImageJ
>> >and Fiji update sites. It is currently the user's responsibility to
>> enable
>> >update sites upon which your update site depends. Some day we may
>> address
>> >that issue as the number of update sites continues to grow, but it is
>> not
>> a
>> >simple feature to add.
>> >
>> >> How does the updater try to determine depencies of plugins? In our
>> >> case while checking our local jars it claimed to detect a cyclic
>> >> dependency which was not there.
>> >
>> >It uses byte-code analysis to detect the dependencies. However, you can
>> >tweak them before uploading by editing the list of dependencies directly
>> in
>> >the right-hand text box of the graphical updater window. This is
>> sometimes
>> >necessary in cases where there are undesired circular dependencies
>> (e.g.,
>> >slf4j-api and various slf4j bindings always detect as circular, due to
>> the
>> >design of SLF4J).
>> >
>> >Regards,
>> >Curtis
>> >
>> >[1]
>> >
>> https://github.com/imagej/imagej-updater/blob/imagej-updater-0.7.1/src/test/java/net/imagej/updater/MultipleSitesTest.java
>> >
>> >On Sun, Mar 15, 2015 at 7:34 AM, Birgit Möller <
>> >

> birgit.moeller@.uni-halle

>> wrote:
>> >
>> >> Dear all,
>> >> we are trying to set up an ImageJ update site for our plugin
>> collection
>> >> MiToBo. Since we have a lot of dependencies we would like to test the
>> setup
>> >> first on our own local server, but in the end we plan to provide
>> Mitobo
>> via
>> >> an ImageJ Wiki update site. Unfortunately we encountered some
>> problems.
>> >> Amongst others we depend on two jars, sezpoz.jar and xalan.jar. Both
>> jars
>> >> seem to have been provided by the Fiji Update Site in former days, but
>> are
>> >> now declared obsolete. Since we need both of them we tried to upload
>> them
>> >> to our own site, however, this fails. First Fiji claims about changed
>> >> checksums for both jar files. After recalculating the checksums and
>> trying
>> >> to upload them again, the updater throws the following exception:
>> >>
>> >> Upload failed: java.lang.NoClassDefFoundError:
>> >> org/apache/xml/serializer/TreeWalker
>> >>
>> >> Then the updater can only be canceled and trying to re-run it requires
>> >> recalculation of checksums again. Nevertheless the upload fails again
>> with
>> >> the above exception. Does anyone have an idea why it is not possible
>> to
>> >> upload both jars to our own site? For other jars unseen by Fiji before
>> >> there were no problems.
>> >>
>> >> And finally two general questions:
>> >>
>> >> - What is Fiji's strategy to deal with cases where the same jar in
>> >> different versions is provided by two update sites?
>> >>
>> >> - How does the updater try to determine depencies of plugins? In our
>> case
>> >> while checking our local jars it claimed to detect a cyclic dependency
>> >> which was not there.
>> >>
>> >> Thanks and best regards,
>> >>
>> >>  Birgit
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------
>> >> Dr. rer. nat. Birgit Moeller
>> >>
>> >> Pattern Recognition & Bioinformatics
>> >> Institute of Computer Science / Faculty of Natural Sciences III
>> >> Martin Luther University Halle-Wittenberg
>> >>
>> >> office:     Room 4.12
>> >> phone:      <a href="tel:%2B49%280%29345-55-24745" value="+493455524745" target="_blank">+49(0)345-55-24745
>> >> fax:        <a href="tel:%2B49%280%29345-55-27039" value="+493455527039" target="_blank">+49(0)345-55-27039
>> >> snail mail: Von-Seckendorff-Platz 1, 06120 Halle / Saale (Germany)
>> >> e-mail:

> birgit.moeller@.uni-halle

>> >> www:        http://www.informatik.uni-halle.de/moeller/
>> >>
>> ------------------------------------------------------------------------
>> >>
>> >> --
>> >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>> >>
>> >
>> >--
>> >ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>> --
>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html
>>
>
> _______________________________________________
> ImageJ-devel mailing list

> ImageJ-devel@

> http://imagej.net/mailman/listinfo/imagej-devel





--
View this message in context: http://imagej-devel.54429.x6.nabble.com/Re-Problems-uploading-jars-to-update-site-tp940p941.html
Sent from the ImageJ-devel mailing list archive at Nabble.com.

_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel


_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problems uploading jars to update site

Birgit Moeller
Hi Curtis!

Curtis Rueden wrote
Hi Birgit,

So your main deps are:

> loci:loci_tools:4.4.8
> net.imglib2:imglib2:2.2.0
> net.imglib2:imglib2-algorithms:2.0.0-beta-10

There are a couple of problems.

1) The imglib2-algorithms artifact is obsolete. Since the 2.0.0 final
release of imglib2, the Algorithms library was renamed to imglib2-algorithm
instead, for consistency.

2) Bio-Formats 4.4.x is no longer shipped with Fiji. It currently ships
Bio-Formats 5.0.x, which will soon become 5.1.0 once the OME team finalizes
that version. MiToBo will have compatibility problems with Fiji as long as
it still depends on 4.4.8.

I advise extending pom-imagej (or alternately, import it in your
dependencyManagement section [1]) to inherit its Bill of Materials [2], to
get consistency of versioning. Otherwise, you are just taking a stab in the
dark at getting all the version numbers right. The BOM exists as a promise
to downstream projects that all the versions declared therein will work
together. In general, if you have not already read over the ImageJ
Architecture page [3], I would suggest doing so.

As of this writing, if you extend or import the newest pom-imagej 6.0.0,
you can then add the following dependencies:

> ome:formats-api (for loci.formats.*)
> ome:formats-common (for loci.common.*)
> ome:ome-xml (for ome.xml.*)
> ome:formats-bsd (runtime; for open file formats)
> ome:formats-gpl (runtime; for proprietary file formats)
> net.imglib2:imglib2
> net.imglib2:imglib2-algorithm

I did not bother to specify the version numbers above -- the whole point is
that you would leave off the "<version>...</version>" lines in your POM,
since they are managed by pom-imagej instead.

Happy to help if you need further information or assistance with this!
I changed our POM file as follows. In the dependency management section I added

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>net.imagej</groupId>
        <artifactId>pom-imagej</artifactId>
        <version>6.0.0</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

and in the dependencies themselves I added all the artifacts you suggested. I did not add any version information and in general it seems to work. There was only one problem. We also use functionality from the loci.formats.gui.* package which is not covered by the dependencies you listed above. I found the packages that we need in

<dependency>
   <groupId>ome</groupId>
   <artifactId>bioformats_package</artifactId>
   <version>5.0.4</version>
</dependency>

however, here I had to specify an explicit version. And if I understood you right, this is again an uber-jar probably not the best choice to depend on. Is the package loci.formats.gui.* somewhere else available, or what is the best strategy to resolve these dependencies?

Curtis Rueden wrote
> We currently don't have an SCM being publically accessible

If you are willing to change that, we would be delighted to add more MiToBo
links in the relevant places of our SciJava propaganda. In particular:

* http://scijava.org/
* http://imagej.net/Philosophy

And on the Architecture page too, adding (e.g.) a MiToBo GitHub
organization, pom-mitobo, etc., to the various tables.
Sounds great, we will discuss that option soon.

Curtis Rueden wrote
Currently we do include the MiToBo and Alida logos in our talks when we
mention SciJava module interoperability, but it would be nice to link to
more of your project resources.
Great, thanks! :-)

Kind regards,

 Birgit
Reply | Threaded
Open this post in threaded view
|

Re: Problems uploading jars to update site

Mark Hiner
Hi Birgit,

>however, here I had to specify an explicit version.

This is because we don't explicitly manage the version of the bioformats_package component, since it's an uber-jar as you mentioned. In cases that you do need to specify a manual version, it would be best to use the ${bio-formats.version} property[1] to avoid mis-matched Bio-Formats versions.

>Is the package loci.formats.gui.* somewhere else available, or what is the best strategy to resolve these dependencies?

Yes, all the Bio-Formats code lives in individual components. I believe all of loci.formats.gui is in formats-bsd. Each component has its own Maven entry - you can see a list of them in pom-scijava[2], for example.

When I'm trying to remember which component a particular Bio-formats class is in, I often go to the master branch on GitHub[3], press 't' to start filtering, and type the class name. All of the Bio-Formats code is organized in /component/[component-name] file structure.

Fiji also ships a Find Jar for Class command[4], which is much more flexible than searching GitHub :)

Hope that helps.

Best,
Mark

On Sat, Mar 28, 2015 at 1:42 PM, Birgit Moeller <[hidden email]> wrote:
Hi Curtis!


Curtis Rueden wrote
> Hi Birgit,
>
> So your main deps are:
>
>> loci:loci_tools:4.4.8
>> net.imglib2:imglib2:2.2.0
>> net.imglib2:imglib2-algorithms:2.0.0-beta-10
>
> There are a couple of problems.
>
> 1) The imglib2-algorithms artifact is obsolete. Since the 2.0.0 final
> release of imglib2, the Algorithms library was renamed to
> imglib2-algorithm
> instead, for consistency.
>
> 2) Bio-Formats 4.4.x is no longer shipped with Fiji. It currently ships
> Bio-Formats 5.0.x, which will soon become 5.1.0 once the OME team
> finalizes
> that version. MiToBo will have compatibility problems with Fiji as long as
> it still depends on 4.4.8.
>
> I advise extending pom-imagej (or alternately, import it in your
> dependencyManagement section [1]) to inherit its Bill of Materials [2], to
> get consistency of versioning. Otherwise, you are just taking a stab in
> the
> dark at getting all the version numbers right. The BOM exists as a promise
> to downstream projects that all the versions declared therein will work
> together. In general, if you have not already read over the ImageJ
> Architecture page [3], I would suggest doing so.
>
> As of this writing, if you extend or import the newest pom-imagej 6.0.0,
> you can then add the following dependencies:
>
>> ome:formats-api (for loci.formats.*)
>> ome:formats-common (for loci.common.*)
>> ome:ome-xml (for ome.xml.*)
>> ome:formats-bsd (runtime; for open file formats)
>> ome:formats-gpl (runtime; for proprietary file formats)
>> net.imglib2:imglib2
>> net.imglib2:imglib2-algorithm
>
> I did not bother to specify the version numbers above -- the whole point
> is
> that you would leave off the "
> <version>
> ...
> </version>
> " lines in your POM,
> since they are managed by pom-imagej instead.
>
> Happy to help if you need further information or assistance with this!

I changed our POM file as follows. In the dependency management section I
added

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>net.imagej</groupId>
        <artifactId>pom-imagej</artifactId>
        <version>6.0.0</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

and in the dependencies themselves I added all the artifacts you suggested.
I did not add any version information and in general it seems to work. There
was only one problem. We also use functionality from the loci.formats.gui.*
package which is not covered by the dependencies you listed above. I found
the packages that we need in

<dependency>
   <groupId>ome</groupId>
   <artifactId>bioformats_package</artifactId>
   <version>5.0.4</version>
</dependency>

however, here I had to specify an explicit version. And if I understood you
right, this is again an uber-jar probably not the best choice to depend on.
Is the package loci.formats.gui.* somewhere else available, or what is the
best strategy to resolve these dependencies?


Curtis Rueden wrote
>> We currently don't have an SCM being publically accessible
>
> If you are willing to change that, we would be delighted to add more
> MiToBo
> links in the relevant places of our SciJava propaganda. In particular:
>
> * http://scijava.org/
> * http://imagej.net/Philosophy
>
> And on the Architecture page too, adding (e.g.) a MiToBo GitHub
> organization, pom-mitobo, etc., to the various tables.

Sounds great, we will discuss that option soon.


Curtis Rueden wrote
> Currently we do include the MiToBo and Alida logos in our talks when we
> mention SciJava module interoperability, but it would be nice to link to
> more of your project resources.

Great, thanks! :-)


Curtis Rueden wrote
> Regards,
> Curtis
>
> [1]
> https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies
> [2] http://imagej.net/BOM
> [3] http://imagej.net/Architecture
>
> [...]

Kind regards,

 Birgit




--
View this message in context: http://imagej-devel.54429.x6.nabble.com/Re-Problems-uploading-jars-to-update-site-tp940p954.html
Sent from the ImageJ-devel mailing list archive at Nabble.com.

_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel


_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: Problems uploading jars to update site

Birgit Moeller
Hi Mark!

Mark Hiner wrote
Hi Birgit,

>however, here I had to specify an explicit version.

This is because we don't explicitly manage the version of the
bioformats_package component, since it's an uber-jar as you mentioned. In
cases that you do need to specify a manual version, it would be best to use
the ${bio-formats.version} property[1] to avoid mis-matched Bio-Formats
versions.
Thanks to your hints specifying a manual version is no longer required.

Mark Hiner wrote
>Is the package loci.formats.gui.* somewhere else available, or what is the
best strategy to resolve these dependencies?

Yes, all the Bio-Formats code lives in individual components. I believe all
of loci.formats.gui is in formats-bsd. Each component has its own Maven
entry - you can see a list of them in pom-scijava[2], for example.

When I'm trying to remember which component a particular Bio-formats class
is in, I often go to the master branch on GitHub[3], press 't' to start
filtering, and type the class name. All of the Bio-Formats code is
organized in /component/[component-name] file structure.

Fiji also ships a Find Jar for Class command[4], which is much more
flexible than searching GitHub :)

Hope that helps.

Best,
Mark

[1]
https://github.com/scijava/pom-scijava/blob/pom-scijava-6.1.0/pom.xml#L207
[2]
https://github.com/scijava/pom-scijava/blob/pom-scijava-6.1.0/pom.xml#L443-503
[3] https://github.com/openmicroscopy/bioformats/tree/master
[4]
http://fiji.sc/Tips_for_developers#find_the_.jar_file_containing_a_certain_class
You were right, I found the missing packages in the formats-bsd component. And the key "t" on GitHub is really a great help :-)
Thanks for your advice and best regards,

 Birgit