ImageJ OPS hackathon results

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

ImageJ OPS hackathon results

Curtis Rueden
Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead

_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Curtis Rueden
Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel



_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Jay Warrick
Curtis - Sweet! I like it. I can see myself making small packages of Ops for things we do in our research that we could easily make available for others. It's also a great way for us to reuse capabilities across different JEX functions we create that allows us to share them with the rest of the community instead of just creating static methods hoarded in various "utility classes" in our software, not that we would ever do that :-)

Curtis and everyone else :-) - First of all, thanks to all for their hard work to lay the foundation for this really useful Ops package. 2 things, though, I would appreciate some help with. Although I've looked at most of the Ops and the tutorials on creating and using Ops, I still have a couple questions.

1) When should we use the "Command" style method of doing things where all information is specified using the @Parameter methodology and run via the "run" method, and when should we use the "Function" style of things with a typed input and output "compute" method? Advantages/disadvantages of each? Can you get by with either?

2) I couldn't see how some of the @Parameter objects would be or are injected or set. What is the "sleekest" method for setting these parameters if I wanted to use these Ops in my own program without resorting to setting private Parameter fields accessible etc (e.g., the @Parameter private T threshold;" of the ApplyConstantThreshold.java Op)? Am I forgetting some tool/method for easily injecting/setting Op/Command parameters? It seems like calls to ij.op().<whatever> only pass parameters to compute method and don't do any @Parameter object injection/setting. Am I wrong? Or, eventually, would these Ops have getters and setters. Are getters and setters automatically generated already that I'm not aware of by just looking over the code?

Thanks,

Jay




On Oct 2, 2014, at 12:35 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel




_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Jay Warrick
Hi All, 

After looking at some of the test classes I can see a partial answer to #2 which is that the ops.run(<name>, <args>) method can be used to set both compute method arguments and @Parameter objects via the magic of modules and the OpsMatchingService. However, how are potential ambiguities resolved when trying to set two parameters of the same type? For example, if I had..

@Parameter
double min;

@Parameter
double max;

and I called my Op with ops.run("MyOp", 10.0, 100.0). 

How does this call know which to assign to min and which to max? Order that it is listed in the Op class def and order of args provided to the ops.run() methods? 

Thanks,

Jay

On Oct 2, 2014, at 4:53 PM, Jay Warrick <[hidden email]> wrote:

Curtis - Sweet! I like it. I can see myself making small packages of Ops for things we do in our research that we could easily make available for others. It's also a great way for us to reuse capabilities across different JEX functions we create that allows us to share them with the rest of the community instead of just creating static methods hoarded in various "utility classes" in our software, not that we would ever do that :-)

Curtis and everyone else :-) - First of all, thanks to all for their hard work to lay the foundation for this really useful Ops package. 2 things, though, I would appreciate some help with. Although I've looked at most of the Ops and the tutorials on creating and using Ops, I still have a couple questions.

1) When should we use the "Command" style method of doing things where all information is specified using the @Parameter methodology and run via the "run" method, and when should we use the "Function" style of things with a typed input and output "compute" method? Advantages/disadvantages of each? Can you get by with either?

2) I couldn't see how some of the @Parameter objects would be or are injected or set. What is the "sleekest" method for setting these parameters if I wanted to use these Ops in my own program without resorting to setting private Parameter fields accessible etc (e.g., the @Parameter private T threshold;" of the ApplyConstantThreshold.java Op)? Am I forgetting some tool/method for easily injecting/setting Op/Command parameters? It seems like calls to ij.op().<whatever> only pass parameters to compute method and don't do any @Parameter object injection/setting. Am I wrong? Or, eventually, would these Ops have getters and setters. Are getters and setters automatically generated already that I'm not aware of by just looking over the code?

Thanks,

Jay




On Oct 2, 2014, at 12:35 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel



_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel


_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Curtis Rueden
In reply to this post by Jay Warrick
Hi Jay,

> 1) When should we use the "Command" style method of doing things where
> all information is specified using the @Parameter methodology and run
> via the "run" method, and when should we use the "Function" style of
> things with a typed input and output "compute" method?
> Advantages/disadvantages of each? Can you get by with either?

We started OPS with the "Command" paradigm, then found that for the vast majority of ops, there was one "special" input over which you want to iterate (either per pixel, or in a multi-threaded context, or both), and one main output of the op. That common case is a Function (you extend either AbstractStrictFunction or AbstractOutputFunction, depending on whether you want to force the caller to pass in a preallocated output or not).

In short: use Function if you want your op to usable by the "map" op to execute it iteratively over an entire image (e.g., an Iterable or IterableInterval). Use a plain Op if you don't need that, don't care or are lazy.

As always in programming: model your code after existing code ("when in Rome") for best results.

> 2) I couldn't see how some of the @Parameter objects would be or are
> injected or set.

Calling OpService#run to execute the op automatically finds the best matching op, and then injects the parameter values in the specified order.

> how are potential ambiguities resolved when trying to set two
> parameters of the same type?
...
> Order that it is listed in the Op class def and order of args provided
> to the ops.run() methods? 

Yes. The order defined in the class must match the order of arguments given to the OpService#run method.

Call OpService#help(String) for a full list of ops with the given name, including expected parameters.

Regards,
Curtis

On Thu, Oct 2, 2014 at 4:53 PM, Jay Warrick <[hidden email]> wrote:
Curtis - Sweet! I like it. I can see myself making small packages of Ops for things we do in our research that we could easily make available for others. It's also a great way for us to reuse capabilities across different JEX functions we create that allows us to share them with the rest of the community instead of just creating static methods hoarded in various "utility classes" in our software, not that we would ever do that :-)

Curtis and everyone else :-) - First of all, thanks to all for their hard work to lay the foundation for this really useful Ops package. 2 things, though, I would appreciate some help with. Although I've looked at most of the Ops and the tutorials on creating and using Ops, I still have a couple questions.

1) When should we use the "Command" style method of doing things where all information is specified using the @Parameter methodology and run via the "run" method, and when should we use the "Function" style of things with a typed input and output "compute" method? Advantages/disadvantages of each? Can you get by with either?

2) I couldn't see how some of the @Parameter objects would be or are injected or set. What is the "sleekest" method for setting these parameters if I wanted to use these Ops in my own program without resorting to setting private Parameter fields accessible etc (e.g., the @Parameter private T threshold;" of the ApplyConstantThreshold.java Op)? Am I forgetting some tool/method for easily injecting/setting Op/Command parameters? It seems like calls to ij.op().<whatever> only pass parameters to compute method and don't do any @Parameter object injection/setting. Am I wrong? Or, eventually, would these Ops have getters and setters. Are getters and setters automatically generated already that I'm not aware of by just looking over the code?

Thanks,

Jay




On Oct 2, 2014, at 12:35 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel





_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Jay Warrick
Very helpful. Thanks.

Not that I need this capability, but is there the potential to create a OpService#run method that would take, for example, alternating strings and objects to allow calls to be "more explicit" or "transparent" (i.e., OpService#runExplicit("myOp", "max", max, "min", min) and potentially more extensible in case order of arguments change or there end up being additional optional arguments. Obviously you link yourself to names as opposed to order. I guess there is always a give and take with these things.

I could try and draft up such a method if you think it useful and doesn't go against what you are trying to shoot for.

Cheers,

Jay



On Oct 3, 2014, at 12:39 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> 1) When should we use the "Command" style method of doing things where
> all information is specified using the @Parameter methodology and run
> via the "run" method, and when should we use the "Function" style of
> things with a typed input and output "compute" method?
> Advantages/disadvantages of each? Can you get by with either?

We started OPS with the "Command" paradigm, then found that for the vast majority of ops, there was one "special" input over which you want to iterate (either per pixel, or in a multi-threaded context, or both), and one main output of the op. That common case is a Function (you extend either AbstractStrictFunction or AbstractOutputFunction, depending on whether you want to force the caller to pass in a preallocated output or not).

In short: use Function if you want your op to usable by the "map" op to execute it iteratively over an entire image (e.g., an Iterable or IterableInterval). Use a plain Op if you don't need that, don't care or are lazy.

As always in programming: model your code after existing code ("when in Rome") for best results.

> 2) I couldn't see how some of the @Parameter objects would be or are
> injected or set.

Calling OpService#run to execute the op automatically finds the best matching op, and then injects the parameter values in the specified order.

> how are potential ambiguities resolved when trying to set two
> parameters of the same type?
...
> Order that it is listed in the Op class def and order of args provided
> to the ops.run() methods? 

Yes. The order defined in the class must match the order of arguments given to the OpService#run method.

Call OpService#help(String) for a full list of ops with the given name, including expected parameters.

Regards,
Curtis

On Thu, Oct 2, 2014 at 4:53 PM, Jay Warrick <[hidden email]> wrote:
Curtis - Sweet! I like it. I can see myself making small packages of Ops for things we do in our research that we could easily make available for others. It's also a great way for us to reuse capabilities across different JEX functions we create that allows us to share them with the rest of the community instead of just creating static methods hoarded in various "utility classes" in our software, not that we would ever do that :-)

Curtis and everyone else :-) - First of all, thanks to all for their hard work to lay the foundation for this really useful Ops package. 2 things, though, I would appreciate some help with. Although I've looked at most of the Ops and the tutorials on creating and using Ops, I still have a couple questions.

1) When should we use the "Command" style method of doing things where all information is specified using the @Parameter methodology and run via the "run" method, and when should we use the "Function" style of things with a typed input and output "compute" method? Advantages/disadvantages of each? Can you get by with either?

2) I couldn't see how some of the @Parameter objects would be or are injected or set. What is the "sleekest" method for setting these parameters if I wanted to use these Ops in my own program without resorting to setting private Parameter fields accessible etc (e.g., the @Parameter private T threshold;" of the ApplyConstantThreshold.java Op)? Am I forgetting some tool/method for easily injecting/setting Op/Command parameters? It seems like calls to ij.op().<whatever> only pass parameters to compute method and don't do any @Parameter object injection/setting. Am I wrong? Or, eventually, would these Ops have getters and setters. Are getters and setters automatically generated already that I'm not aware of by just looking over the code?

Thanks,

Jay




On Oct 2, 2014, at 12:35 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel






_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Curtis Rueden
Hi Jay,

> is there the potential to create a OpService#run method that would
> take, for example, alternating strings and objects to allow calls to
> be "more explicit" or "transparent"

All Ops are Modules, and the ModuleService has this feature:


Or more type-safe using a Map<String, Object>:


Regards,
Curtis

On Fri, Oct 3, 2014 at 5:25 PM, Jay Warrick <[hidden email]> wrote:
Very helpful. Thanks.

Not that I need this capability, but is there the potential to create a OpService#run method that would take, for example, alternating strings and objects to allow calls to be "more explicit" or "transparent" (i.e., OpService#runExplicit("myOp", "max", max, "min", min) and potentially more extensible in case order of arguments change or there end up being additional optional arguments. Obviously you link yourself to names as opposed to order. I guess there is always a give and take with these things.

I could try and draft up such a method if you think it useful and doesn't go against what you are trying to shoot for.

Cheers,

Jay



On Oct 3, 2014, at 12:39 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> 1) When should we use the "Command" style method of doing things where
> all information is specified using the @Parameter methodology and run
> via the "run" method, and when should we use the "Function" style of
> things with a typed input and output "compute" method?
> Advantages/disadvantages of each? Can you get by with either?

We started OPS with the "Command" paradigm, then found that for the vast majority of ops, there was one "special" input over which you want to iterate (either per pixel, or in a multi-threaded context, or both), and one main output of the op. That common case is a Function (you extend either AbstractStrictFunction or AbstractOutputFunction, depending on whether you want to force the caller to pass in a preallocated output or not).

In short: use Function if you want your op to usable by the "map" op to execute it iteratively over an entire image (e.g., an Iterable or IterableInterval). Use a plain Op if you don't need that, don't care or are lazy.

As always in programming: model your code after existing code ("when in Rome") for best results.

> 2) I couldn't see how some of the @Parameter objects would be or are
> injected or set.

Calling OpService#run to execute the op automatically finds the best matching op, and then injects the parameter values in the specified order.

> how are potential ambiguities resolved when trying to set two
> parameters of the same type?
...
> Order that it is listed in the Op class def and order of args provided
> to the ops.run() methods? 

Yes. The order defined in the class must match the order of arguments given to the OpService#run method.

Call OpService#help(String) for a full list of ops with the given name, including expected parameters.

Regards,
Curtis

On Thu, Oct 2, 2014 at 4:53 PM, Jay Warrick <[hidden email]> wrote:
Curtis - Sweet! I like it. I can see myself making small packages of Ops for things we do in our research that we could easily make available for others. It's also a great way for us to reuse capabilities across different JEX functions we create that allows us to share them with the rest of the community instead of just creating static methods hoarded in various "utility classes" in our software, not that we would ever do that :-)

Curtis and everyone else :-) - First of all, thanks to all for their hard work to lay the foundation for this really useful Ops package. 2 things, though, I would appreciate some help with. Although I've looked at most of the Ops and the tutorials on creating and using Ops, I still have a couple questions.

1) When should we use the "Command" style method of doing things where all information is specified using the @Parameter methodology and run via the "run" method, and when should we use the "Function" style of things with a typed input and output "compute" method? Advantages/disadvantages of each? Can you get by with either?

2) I couldn't see how some of the @Parameter objects would be or are injected or set. What is the "sleekest" method for setting these parameters if I wanted to use these Ops in my own program without resorting to setting private Parameter fields accessible etc (e.g., the @Parameter private T threshold;" of the ApplyConstantThreshold.java Op)? Am I forgetting some tool/method for easily injecting/setting Op/Command parameters? It seems like calls to ij.op().<whatever> only pass parameters to compute method and don't do any @Parameter object injection/setting. Am I wrong? Or, eventually, would these Ops have getters and setters. Are getters and setters automatically generated already that I'm not aware of by just looking over the code?

Thanks,

Jay




On Oct 2, 2014, at 12:35 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel







_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel
Reply | Threaded
Open this post in threaded view
|

Re: ImageJ OPS hackathon results

Jay Warrick
Awesome. Forgot this. Thanks!

J

On Oct 3, 2014, at 5:36 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> is there the potential to create a OpService#run method that would
> take, for example, alternating strings and objects to allow calls to
> be "more explicit" or "transparent"

All Ops are Modules, and the ModuleService has this feature:


Or more type-safe using a Map<String, Object>:


Regards,
Curtis

On Fri, Oct 3, 2014 at 5:25 PM, Jay Warrick <[hidden email]> wrote:
Very helpful. Thanks.

Not that I need this capability, but is there the potential to create a OpService#run method that would take, for example, alternating strings and objects to allow calls to be "more explicit" or "transparent" (i.e., OpService#runExplicit("myOp", "max", max, "min", min) and potentially more extensible in case order of arguments change or there end up being additional optional arguments. Obviously you link yourself to names as opposed to order. I guess there is always a give and take with these things.

I could try and draft up such a method if you think it useful and doesn't go against what you are trying to shoot for.

Cheers,

Jay



On Oct 3, 2014, at 12:39 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> 1) When should we use the "Command" style method of doing things where
> all information is specified using the @Parameter methodology and run
> via the "run" method, and when should we use the "Function" style of
> things with a typed input and output "compute" method?
> Advantages/disadvantages of each? Can you get by with either?

We started OPS with the "Command" paradigm, then found that for the vast majority of ops, there was one "special" input over which you want to iterate (either per pixel, or in a multi-threaded context, or both), and one main output of the op. That common case is a Function (you extend either AbstractStrictFunction or AbstractOutputFunction, depending on whether you want to force the caller to pass in a preallocated output or not).

In short: use Function if you want your op to usable by the "map" op to execute it iteratively over an entire image (e.g., an Iterable or IterableInterval). Use a plain Op if you don't need that, don't care or are lazy.

As always in programming: model your code after existing code ("when in Rome") for best results.

> 2) I couldn't see how some of the @Parameter objects would be or are
> injected or set.

Calling OpService#run to execute the op automatically finds the best matching op, and then injects the parameter values in the specified order.

> how are potential ambiguities resolved when trying to set two
> parameters of the same type?
...
> Order that it is listed in the Op class def and order of args provided
> to the ops.run() methods? 

Yes. The order defined in the class must match the order of arguments given to the OpService#run method.

Call OpService#help(String) for a full list of ops with the given name, including expected parameters.

Regards,
Curtis

On Thu, Oct 2, 2014 at 4:53 PM, Jay Warrick <[hidden email]> wrote:
Curtis - Sweet! I like it. I can see myself making small packages of Ops for things we do in our research that we could easily make available for others. It's also a great way for us to reuse capabilities across different JEX functions we create that allows us to share them with the rest of the community instead of just creating static methods hoarded in various "utility classes" in our software, not that we would ever do that :-)

Curtis and everyone else :-) - First of all, thanks to all for their hard work to lay the foundation for this really useful Ops package. 2 things, though, I would appreciate some help with. Although I've looked at most of the Ops and the tutorials on creating and using Ops, I still have a couple questions.

1) When should we use the "Command" style method of doing things where all information is specified using the @Parameter methodology and run via the "run" method, and when should we use the "Function" style of things with a typed input and output "compute" method? Advantages/disadvantages of each? Can you get by with either?

2) I couldn't see how some of the @Parameter objects would be or are injected or set. What is the "sleekest" method for setting these parameters if I wanted to use these Ops in my own program without resorting to setting private Parameter fields accessible etc (e.g., the @Parameter private T threshold;" of the ApplyConstantThreshold.java Op)? Am I forgetting some tool/method for easily injecting/setting Op/Command parameters? It seems like calls to ij.op().<whatever> only pass parameters to compute method and don't do any @Parameter object injection/setting. Am I wrong? Or, eventually, would these Ops have getters and setters. Are getters and setters automatically generated already that I'm not aware of by just looking over the code?

Thanks,

Jay




On Oct 2, 2014, at 12:35 PM, Curtis Rueden <[hidden email]> wrote:

Hi Jay,

> Am I right that Ops sort of occupies the niche between ImgLib2 and
> ImageJ Plugins... something that makes it easier to do the image
> manipulations but can be reused a bit more easily given they don't
> require many of the Service parameters and preprocessors that many of
> the plugins take/need?

Yes, OPS is intended for pure image processing operations and functions. The rule of thumb is that they be deterministic, and have no side effects. So you give same inputs, you get same outputs, every time. Many of them are also multithreadable, though that is not a requirement. And OPS are also supposed to be "static" rather than dynamic -- i.e., they shouldn't have a variable number of input or output parameters, unlike commands in general.

That said, OPS are still allowed to depend on services, but it is expected that the service methods you call will not compromise the determinism of the op -- i.e., only utility methods of services should really be used. Perhaps in the future we could add annotations to each service method indicating what sort of method it is, and hence where it is "safe" to use.

I want to thank you for your feedback and discussion from a few months ago, regarding reuse of ImageJ2 commands in JEX. Your perspective provided some of the inspiration for the design of OPS, because it became clear that we need a "pure functional" layer for image processing that does not rely on side effects from services, etc. The idea is that KNIME Image Processing, CellProfiler, OMERO, JEX, etc., can all consume and expose the ops with the assumption that they will behave well (work headless, etc.).

Regards,
Curtis


On Thu, Oct 2, 2014 at 12:08 PM, Jay Warrick <[hidden email]> wrote:
Looks promising. Am I right that Ops sort of occupies the niche between ImgLib2 and ImageJ Plugins... something that makes it easier to do the image manipulations but can be reused a bit more easily given they don't require many of the Service parameters and preprocessors that many of the plugins take/need?

On Oct 1, 2014, at 4:58 PM, Curtis Rueden <[hidden email]> wrote:

Hi everyone,

The ImageJ2 and KNIME Image Processing teams met in Madison during the week of September 15 - 19, to work on ImageJ OPS, which seeks to be a unifying library for scientific image processing.

On behalf of the OPS development team, I am pleased to announce the results of that hackathon, including accomplishments, project goals and milestones. See the news post for full details:


Regards,
Curtis Rueden
ImageJ2 project lead
_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel







_______________________________________________
ImageJ-devel mailing list
[hidden email]
http://imagej.net/mailman/listinfo/imagej-devel