Plugin/theme updates directory integration

Projects Core Development
1

@Simone and I have been working on the directory and the core integration plugin. We would like some feedback and advice on the best way to handle plugin/theme updates as it relates to plugins/themes that may be listed in both the WordPress repo and ClassicPress directory.

Simone’s initial approach to handling updates was to make Update URI header required, which would contain the directory API endpoint with a slug:

Update URI: https://directory.classicpress.net/wp-json/wp/v2/plugins?byslug=classicpress-directory-integration

If a plugin is listed only in the CP directory, there is no problem. However, plugins/themes may be listed in WP repo. We want those plugins/themes to get listed in CP directory, so our users have access.

The problem is WP repo, like CP directory, doesn’t allow third-party Update URI headers. So if we require it, plugins/themes would violate WP repo rules. Developers won’t maintain two separate versions of their plugins/themes, so we need a better way to handle updates without requiring Update URI header.

The header was initially solving the problem of having plugins/themes listed in both places (WP/CP), so CP core knows where the updates are coming from. If a user installs a plugin from CP directory, but there is a plugin in WP repo with the same slug, where will the update come from? Where should it come from? And vice versa.

Simone mentioned that without using Update URI, the core update code may need changes.

My only idea to try and solve this issue is to track the source of installation in the database and then check it during updates and handle updates correctly based on the source of the installation. But this does require core changes. Tracking the source of installation in the database will also allow us to add some sort of a visual identifier to plugins/themes to show the user where the item was installed from (a badge, a column with source, etc.). This is just an idea.

I think the most important things to keep in mind trying to solve this problem are:

  • Ensure there are no conflicts between plugins/themes with the same slug if they are listed in both WP repo and CP directory
  • We can’t require something that would prevent WP plugins/themes from being listed in CP directory
  • Ensure the update process is seamless for the user

We would like some feedback on the best way to handle it.

Thoughts @timkaye @MattyRob ?

1 Like
2

Can this be done with an if statement? I think the Update URI is different in CP and WP, so when a check on plugins is made can it include a check on the format of the Update URI that says if the Update URI is CP then it handles the plugin/theme from the CP dir if it is WP it handles it from the WP repos?

I can understand this is a change in core and it would be better not to touch the plugin/theme install/update/delete process in core to be able to make use of WP repos as much as possible (because once our ecosystem is grown up we might not need them anymore) but would it cause trouble putting this check in place before any action happens with plugins and themes?

EDITED TO ADD: my idea takes into account the use of Update URI and we prefer not to include it because it might cause plugin to not be listed in wp repo. What if plugin/theme slug is prefixed with something if it comes from CP? can this be done?

1 Like
3

It’s good to see that you are finally addressing my original concern with the Update URI code.
I think the problem is in thinking that a plugin should be in both the WP repo and the CP directory.
If a plugin is already in WP repo, and wants to support CP, it can do that from the WP repo. It is not necessary to be in the CP directory as well. But perhaps the CP directory should be able to point to an entry in the WP repo instead of just to GitHub repos.

We have to consider that the Update URI code was to solve a problem with WP repo versus external repos. A plugin without a value defaults to WP repo. Using the code verbatim as we have, it takes on a different meaning. CP can’t use the no-value to mean CP directory unless it does a double check, by checking CP directory first and then WP repo if it’s not found. Even then, it could get the wrong version, but then if there isn’t a value for the URI, that should be the expected result.

4

That’s an interesting suggestion. I’d love to see what others think.

The way I see this happening, if this is the route we want to go with:

  1. Directory can take WP repo as a source, asking for a slug.
  2. Plugin/themes go through the same review process.
  3. We can use WP.org plugin/theme API (just like GitHub API) to keep listings updated. For example:
    • https://api.wordpress.org/plugins/info/1.0/beaver-builder-lite-version.json
  4. On the core side, if WP slug is present during update check in the API response, the core would handle it normally. If it’s not present, CP only plugin, then our own directory update integration would engage.

This would remove at least one barrier to entry for WordPress plugins/themes, as not everyone uses GitHub.

Thoughts?

1 Like
5

If you allow themes and plugins to be hosted at WP, then you will have two completely different systems for monitoring them, run by completely different teams. So a guidelines violation or security problem could be dealt with very differently, simply because of where it is hosted. That’s the very definition of double standards.

1 Like
6

I’m not sure I know enough about the Directory to hold a useful or valid opinion here but…

WordPress actually host the code. For CP are we planning the same or are we just listing plugins and themes that work on ClassicPress?

If the latter of these then whatever we do need to support code hosted on WP and anywhere else for that matter.

It seems that the Update URI header should do that and it could point to WP for our purposes.

1 Like
7

I would prefer to avoid it, we need a workable solution to manage plugins in the core from both sources without conflicts and requiring headers that would violate WP repo rules.

We’re just listing them and providing an API for the core. Directory simply provides data core requests. You can see an example request:
https://staging-directory.classicpress.net/wp-json/wp/v2/plugins?byslug=subscribe2-for-cp

So far we’ve used similar approach to Beda’s plugin, even tried Beda’s plugin with the new API. This means we have:

  • A dedicated plugin search page (themes will be added too)
  • Once you install plugin, it shows up on your regular Plugins page
  • And the experimental integration requires Update URI header to work (which is a problem because WP repo-listed plugins can’t have third-party URIs).

The problem is finding a way for WP and CP plugins to co-exist together without conflicts, especially when it comes to updates and identical slugs.

8

I don’t understand this. You say in response to me that you don’t want plugins or themes hosted at WP, but now you’re saying you want a way for this to happen.

As I said previously, if you permit plugins and themes to be hosted from WP, then you’ll have different standards applied to them in comparison to those listed purely for CP. I don’t see how that’s defensible.

9

No, WP and CP plugins need to be able to work without conflict inside CP core.

WP-hosted plugins listed in the directory was in response to Joy’s suggestion.

Ideally, if a WP plugin wants to be listed in the directory, it will go through the same process as a CP plugin: Have a GitHub repo, submit the plugin, get reviewed, and get listed.

My initial post specifically focuses on plugins listed in the directory with a GitHub repo attached, it’s not related to anything hosted in WP repo.

But, the problem is when plugins/themes are both in WP repo and CP directory. For example:

If we require Update URI for plugins listed in the directory for updates to work, then the plugin would violate WP repo rules and get suspended. Developers will not maintain 2 separate plugins.

But without Update URI, we haven’t found a workable solution to update plugins without needing changes to the core. That’s what this post is about.

You can ignore my post about WP-hosted plugins and integrating WP.org API. That was just an idea in response to another suggestion.

10

With CP v.2 we run into the problem that there is not a definite way to say that a plugin is CP compatible.
With v.1 compatibility with WP 4.9 is the way, but now any plugin can be compatible but can also break calling block related functions.

Can this be a solution?

  • list CP plugins as “vetted”, require Update URI, require GitHub repo, review them, serve updates.
  • list WP plugins that wants also be in our directory (in this case the developer is up to the developer to check block function existence before calling them), declare that those are reviewed at WP, require Require CP header, install them from the directory, updates served by WP.
11

I must be missing something. If we require CP list plugins to have the ‘Update URI’ plugins listed at WordPress can add the header pointing to the WP repository.

This seems to say the header isn’t needed but it doesn’t say it’s not allowed.

12

It’s not allowed to point out from WP.
That header is useful for CP plugins for two reasons:

  • it prevents a plugin to be overwritten by another in the WP repo that have the same slug
  • it let us use the builtin CP update system that handles plugins outside WP repo (update_plugins_{$hostname} filter)
13

So this issue is isolated then to plugins hosted at WP but that also wish to be listed in the ClassicPress directory?

Are there any such plugins? If the plugin supports both and is in the WP repository are we not just confusing things by listing it ourselves?

With such plugins they willet updated from the WP repository won’t they, so the header us redundant. Would this be resolved by requiring use of the Update URI header for all themes and plugins hosted anywhere other than on WP?

14

Update URI was added to allow non-repository plugins to be updated from their own sources. But to be listed in the WP repo, they must follow the rules which includes “Plugins may not send executable code via third-party systems.”

Yes.

The Laravel directory (current one) has a few plugins listed that link to WP.org repo. But with the new directory, they will not be allowed unless they go through the submission process.

My goal once we launch the new directory is to contact plugin/theme developers that state compatibility with ClassicPress in WP.org repo to get them to become listed in our directory. Our directory will be a ghost town without getting WP plugins to be listed. We don’t have enough plugin/theme developers yet.

If we don’t want WP plugins to be listed in our directory, why even have a directory? We could ask CP developers to submit to WP and let that be the only place for plugins/themes.

There will be a point in the future where we have enough plugins/themes were we will be comfortable enough to say “let’s remove WP repo integration from the core” so we can have full control over CP ecosystem and our users don’t need search through hundreds of block plugins to find something compatible with CP. But we need to build that ecosystem, which means getting plugins/themes from WP listed in our directory.

I don’t see CP directory becoming successful without having WP plugins/themes listed in there. #1 complaint from CP users and outsiders looking in is lack of plugins/themes. This is how we can fix it, and I plan on actively engaging developers to get them to join directory.

This is why, at this point, we can’t require Update URI for updates.

1 Like
15

We can’t ask them to unlist from WP. This is for sure.
And we can’t require standards that force them to be thrown out of WP repos.

I have a question however, there was a discussion once about enabling more sources to download and update plugins in the dir. At present we require GitHub.

Can we require SVN or GitHub?

That way we could say: If you have a plugin listed in the wp repo, you can list it in the CP dir by providing the SVN source.

In time, all the VERY OLD PLUGINS compatible with WP 4.9 will stop working with WP, and these devs when the WP repo deletes them will have the chance of putting them on GitHub and updating the update source on the dir by giving the GitHub source.

Is that feasible? so if the plugin has a svn source it gets taken from there and if a GitHub one it comes from GitHub.

16

Is there any drawback in having (as in the laravel dir) “developed for CP” and “works in CP” or similar?

I’m considering another point. We are developing directory integration as a plugin. And this plugin works on WP and can give CP developers a way to push their plugin to WP users.

17

My original point was that because of how the header was added late in the game, plugins that don’t have a value for Update URI will default to the WP repo. If CP code changes that to default to the CP directory, it has to do a double check for the majority (or not find them at all).
I would suggest that the review process only allows URIs for the CP directory or the WP repo, (depending on how you handle blanks) so that there are no security holes for naive users.

1 Like
18

I’m unsure if there’s any value to having those 2 separated. Unless it’s to help filter for CP plugins if the integration plugin is installed on a WP site.

This is why I think it would be valuable to track the source of the installation. The default value for Update URI would be determined by what the installation source is. Let’s say it’s installation_source: cp|wp:

  • When an update is initiated, it checks the installation_source:
    • If Update URI has no value and wp, it defaults to WP.org.
    • If Update URI has no value and cp, it defaults to CP directory.
    • If Update URI has a value set, it uses the URI set.

The core would simply track installation_source when the plugin is installed in the options table. And then perform the check if Update URI is not set.

I think this is the best way to enable both sources without requiring Update URI set.

Would the above process satisfy this?

19

No, your logic only extends the checks to another field. Since that field doesn’t exist, you have to have a default of wp so you are back to where you started.
As for the security holes opened by Update URI, it is totally unrelated. I’m advocating that the CP directory use the same review standard as WP repo, and not allow transmission of code (installing from third party site) by limiting the accepted Update URI to CP dir or WP repo. The naive user doesn’t realize that a lax developer’s site could be hacked and the plugin could be replaced by malware. Since the code is open source, the use of Update Manager or similar can be a magnet for hackers to target the site that hosts a plugin like that. I’m not saying that people shouldn’t write plugins that self-update. I’m saying that CP directory should not list them.

1 Like
20

That’s already given. We’re trying to find a way for plugins/themes to update from the source they were installed, which is either WP repo or CP directory if Update URI is empty. If Update URI is set, that will be used. Directory will not list plugins with a third-party URI.

My suggestion of tracking installation source requires core changes, where we can ensure. We set the defaults in the core, so the core knows what to do. wp will have its own default, cp will have its own default. That’s what we need to do, make necessary core changes. I’m not seeing security issues.

If user installs a premium plugin that sets Update URI that’s their choice. The core will respect it and use it. The premium plugin will not be listed in the directory.