Add an extension mechanism for loading custom filters

[mallman]

I've developed several custom filters for my own use, and it would be helpful to have something like CIFilterBox for near-time feedback during development. IMHO this would multiply the value of CIFilterBox for developers.

There doesn't seem to be a consensus or standard mechanism for package filters at this point in time. There is the Image Unit, but it is an old API that Apple seems to discourage. I gather that from reading the API documentation and the archived documentation.

Swift Packages with Core Image filters seem to be a newish thing, if non-standard. See https://forums.swift.org/t/swift-package-with-core-image-ci-metal-resources/40962/5, where I'm hoping to get some info on a SPM plugin for a packaging custom filters in a Swift package.

Currently I'm building my external filters as custom Frameworks, which is entirely non-standard. This works for me presently, because I do not share these publicly. If CIFilterBox supports a common mechanism (or two) for loading custom filters, I would look at that as a way to test my filters.

[DavidWingate]

So I've had a little look into this one, and it's a lot harder than your other suggestions! (Not least because I'm not actually a core image developer). The CIFilterBox UI is dynamically generated according to the contents of the filter's attributes dictionary, and since I have no idea what's in a custom filter's dictionary (if anything), there would have to be a lot of fallbacks and potentially no input controls would appear.

Perhaps if you wouldn't mind sending or pointing me toward one of your custom filters for testing? Maybe I can work out a simpler method that way. For what it's worth, I would probably start down the 'Image Unit' route to begin with. It may be old, but until Apple picks a new winner and documents it, the Image Unit still seems to be the standard.

For the moment at least, I think this feature is quite a long way off!

[mallman]

Hi David,

I've spent some more time working with core image filters (and more general metal shaders) over the past few weeks. What I'm finding is it's barely possible with SPM, but I wouldn't call it a robust or "standard" approach. By that I mean, it's possible to do, but how you do it won't necessarily agree with the way other people do it. I think the SPM approach is still too immature to consider.

I've looked into Apple's Image Units briefly. I agree this appears to be Apple's "standard" for sharing core image filters. As you noted, it's a bit dusty and unloved. As you can see from the API documentation, it looks like Apple is deprecating support for executable filters—probably for security reasons. Removing this support would be crippling. I'm not even sure how Apple will feel about distributing an app that uses these APIs. If I were you, I wouldn't want to invest a lot of time integrating this capability only for Apple to reject my app. Maybe you can send them a question.

I haven't used Image Units myself. I've been using a combination of my own approach to SPM filters and custom Frameworks.

BTW, I want to comment on one specific point you made:

I have no idea what's in a custom filter's dictionary

Actually, a properly packaged custom filter should include its own attribute dictionary. You should be able to introspect on this as you would built-in filters.

Perhaps if you wouldn't mind sending or pointing me toward one of your custom filters for testing?

Let me try converting one of my Framework-based filters to an image unit. I will share my lessons and my code if I'm successful. Note I probably won't get to this for a while.

For the moment at least, I think this feature is quite a long way off!

Agreed. Thanks for your consideration.