Draft of the Plugin Directory Submission Guidelines

[azurecurve]

With the Plugin Directory now having plugins added, it’s probably time to finalise the guidelines. Below is a draft of the plugin submission guidelines.

---

In order to submit a plugin to the ClassicPress Plugin Directory, please follow these steps:

1. Register on the ClassicPress Forum; if you will be submitting plugins on behalf of a company, use an official company email.
2. Navigate to the ClassicPress Plugin Directory and sign in using your ClassicPress Forum user.
3. Click your name/avatar in the top right corner and select Create Develop Profile.
4. Navigate to your developer profile and click the New Plugin button.
5. Complete all fields on the New Plugin page, being careful to select the correct Plugin Type; the plugin repository on GitHub must contain a complete, final, version.
6. Once plugins have been submitted, they will be queued for review; submitted plugins must adhere to the Plugin Directory Guidelines.

If any issues with the submitted plugin is found, the Plugin Directory moderators will contact the developer(s) and work with them towards a resolution.

Once approved a plugin will appear in the ClassicPress Plugin Directory.

[wadestriebel]

This is great, thank you for starting the drafts of these

[anon66243189]

6. Once plugins have been submitted, they will be queued for review; submitted plugins must adhere to the Plugin Directory Guidelines.

This point is probably the most important of all - since it will avoid bad code being shared on CP as stable, safe plugin (or theme).

Do we have designated people to do this?
This is a huge amount of work. I know from personal experience just looking at the feedback I get from WP Plugin Review team.
The main things to look for are sanitise, escape and capabilities, but this takes time.

Just as an example, my own plugin I submitted a few days ago was accepted here very quickly.
However, it didn’t follow these 3 “rules” (sanitise, escape, capabilities) properly in all aspects.
The plugin is now updated to a v 2.0.0 (on git) because of the feedback of WP Review team, to which I also submitted the plugin, and thus it is now safe.

However we want to trust a Developer does the right thing, actually humans can make (and do make) mistakes, and that is why review has to be done with the same philosophy as we sanitise/escape/capabilities:
Do not trust, anyone.

I think we need to be much more careful and harder here, then we are right now.

Given we likely do not have the *-power to do this, I would suggest we remove the “developed for CP” and only allow “Works with CP”, and force plugins to be first accepted within WP Core.
Unless we can grant a full review, doing otherwise will bite us in the butt very soon, because we will have unsafe stuff online.

---

I have seen there are people active in the security business present on this forum, but I am not sure how much they are willing to contribute to plugin review in a volunteer way.
That could, of course, resolve this issue, as we would have our “own” review team or person.

[wadestriebel]

@anon66243189

I would suggest we remove the “developed for CP” and only allow “Works with CP”, and force plugins to be first accepted within WP Core.

I personally don’t like this approach, I think in order for the community to survive we need to have our own plugins. By continuing to rely on plugins being submitted to WP we are thereby undermining ourselves.

I do think we need a committee to review the plugins, particularly people who have experience doing so. I think to start it would be nice to have a group of plugin devs that are currently active in the community help review those in an official capacity.

[anon66243189]

I absolutely agree, and maybe I explained my thought wrong.

Totally, the CP needs to distance itself from WP and relying on their plugins of course is a death sentence so to speak.
What I mean, is that until we have a security team, we perhaps should not accept non-revised plugins.

This because a security issue in one of “our” plugins will likely hurt CP more than “relying” on actual safe plugins.

Absolutely agree that on the (possibly Short, but more likely Long) term we need our own Plugins.
We just need a method to ensure 100% safety (well, there is no such thing as 100% safety but we can strive towards) on CP Plugins, and that is only possible with continuous and strong review.

Note, in WP the reviewers are actually paid volunteers more often than not (they work for companies who pay them to volunteer for WP).
Thus of course it is easy for WP to review plugins - less easy for CP because we do not pay volunteers and have no companies sponsoring us with “volunteers”.

This is the entire reasoning behind my suggestion to lock plugin submission without review until we have a real review (referring to code review, not guidelines review).

Hope that clarifies it

[LinasSimonis]

@anon66243189

until we have a security team, we perhaps should not accept non-revised plugins.

What about more democracy to this approach?

I think, that an ideal solution can be the labelling. I see 3 kinds of labels:

“This plugin is not reviewed by any security team”

“This plugin is reviewed by the CP security team for basic security vulnerabilities”

“This plugin is reviewed by xxx for xxx security vulnerabilities.”

Inform people, but don’t make decisions. In other words, options, not decisions.

And yes, the third-party option can be paid (not sure, @pluginvulns have this option, but i’m sure, there will be a few players). But this is the plugin’s owner’s decision.

[anon66243189]

Off topic but perhaps useful for “review plugins” topic:

I’ve just finished setting up phpcs (with wpcs) on my machine and from now on the basic things won’t escape me anymore

I can recommend this tool to anyone coding and with it installed, review team has a much easier task, as it recognizes many things (non escaped variables for example or Yoda conditions not respected) automatically, which increases safety a lot.

[azurecurve]

@wadestriebel

committee to review the plugins, particularly people who have experience doing so

I don’t have experience reviewing plugins, but happy to be involved in this. Obviously, I’d need to recuse myself when my plugins are involved (it seems I also have a backlog of plugins to get submitted).

[azurecurve]

@anon66243189

feedback I get from WP Plugin Review team.

When I was submitting plugins to WP, I had virtually no feedback from the WP plugin team. I know from the rewrites I did for ClassicPress that I had quite a few small security issues in plugins which were never picked up by the WP team.

[azurecurve]

A decision needs to be taken on the level of reviewing which will/can be done and then after how it is presented in the directory.

I can take a look at the tool @anon66243189 mentioned as it does sound like it will be useful.

[wadestriebel]

Thank you, maybe let’s set up a community meeting for the week of July 5th and we can finalize this and get it moving forward

Doodle Poll: https://doodle.com/poll/isg5fa6zfa8twwh3

[Paul]

@anon66243189

This because a security issue in one of “our” plugins will likely hurt CP more than “relying” on actual safe plugins.

Preventing plugins into the directory because of a fear of security vulnerabilities is going to really hinder progress here.

Guidelines are there to guide developers and requirements for submission are there for developers to ensure their plugins meet certain minimum standards.

Good security practice is about clear education and experience.

The striving toward 100% security, or near to, is an effort in futility and will burn resources very quickly - and by resources I mean people. The enormous size of the codebase of some plugins makes it untenable to have reviews “by committee” - if you’re going to set the bar to be “as near to 100% safe as possible”, the testing for such a thing will always need to be automated.

Even then, you’re going to have security vulnerabilities. It’s the cost of doing business. A security vulnerability doesn’t “hurt” reputation in the long term really… it’s the response and handling of the vulnerability that really matters.

I feel in this discussion there’s a huge over-emphasis on security requirements and labelling, and choices/options etc. Developers vary in their competency, and so does their security handling within their code. There’s no getting around that. The WordPress plugin directory, while it has its flaws in some areas, gets the job done really well. It’s open to anyone to create and contribute and there’s no labels like “not reviewed” or “basic vulnerability reviewed”. Everyone accepts the risk that a plugin has undergone certain basic tests, or maybe not, and they get on with it.

If you want people to contribute and get involved, making the barrier to entry so high and adding complexity will defeat your purpose.

If you’re going to put requirements in place, you’ll need to make it very clear

• clear documentation on what they are
• how to meet them
• what to do if things don’t pass
• processes for updating the requirements and communicating updates.

A simple example of where none of this applies is the plugin submissions form/process.

I tried for the 2nd time to submit a plugin, using a dead link/form that’s been clearly non-functional for weeks. So I’m wasting my time doing that, then submitting forums requests, then digging around to see if I can find answers, the discovering that the guidelines to submissions are buried in the forums.

And it could have been avoided by having the dead link updated with a link to brief outline of the current guidelines and linking to the live submissions page.

My points is… if you’re going make rules and policies for developers to contribute, document and make it clear how things work before implementing them. Forums aren’t a substitute for documentation and clear guidelines.

[pluginvulns]

@azurecurve

When I was submitting plugins to WP, I had virtually no feedback from the WP plugin team. I know from the rewrites I did for ClassicPress that I had quite a few small security issues in plugins which were never picked up by the WP team.

We can provide some additional insight on that, which might help with decisions being made on in how this is handled with ClassicPress.

Currently, the team running the WordPress Plugin Directory is listed as having only four members. So while it is claimed that a manual review is done of each plugin, which includes checking the security, as a practical matter, that doesn’t seem like it could be happening. Other things we have seen point to them not happening in a lot of instances as well.

If the security review portion of that is happening, the results are not good, as two weeks ago we spotted two new plugins being added to their directory that contained serious vulnerabilities, which hackers would exploit. We spotted the possibility of those through an automated tool, which we have offered for years to provide them access to or offered to help them have similar capabilities, so there isn’t an excuse for that. The same week we spotted another recently added plugin that has serious vulnerability, which hackers would exploit, that a manual review should have caught, and our automated tool caught when the developer fixed the broken state of the code (which allowed the vulnerability to be exploited).

In other cases, when we have gone to check on new plugins with possible security issues, we couldn’t figure out if they were vulnerable because the plugin was too broken. That points the review not always happening and also an indication of the state you might find plugins being submitted to be in.

While having 100% security is nearly impossible, it wouldn’t be hard to provide better results than what WordPress does with plugins and what, unfortunately and troublingly, much of the security industry around WordPress seems to consider acceptable.

Even limited checks to see if basic security is in place, to check for the possibility of serious issues, or some combination of those two, would be a big improvement over what has WordPress done. As we mentioned in another topic, we can provide free access to our Plugin Security Checker, which flags the possibility of plenty of that. It requires some security knowledge to check over the results and certainly won’t catch everything, but it can help provide better results than WordPress is delivering. When we ran the plugins currently in your directory through that we found multiple plugins with vulnerabilities, including two WordPress plugins that had the vulnerabilities in them since their first versions, the version that was supposed to have been manually checked by the WordPress team.

[azurecurve]

@pluginvulns

we can provide free access to our Plugin Security Checker

I think this would be very useful; there is a meeting next week re: plugin submissions; I’ll make sure this offer gets a mention.

We nee dot find a line for security between @anon66243189 and @Paul where we encourage security, but not make the cost of entry too high. Not in quite the same way, but I think this is what WP is now doing with blocks and React; smaller/new developers won’t be able to access the marketplace in the same way as previously which will restrict competition and isn;t necessarily going to drive standards up.

[anon66243189]

I absolutely agree that a bar cannot be set so high that no one can cross it.
And yet, this is CPs’ catchphrase:
Stable. Secure. Instantly Familiar.

Secure.
I think CP (should) tries to make a difference to WP here.
Secure does not only mean “an update wont break your site” (which no WP update ever really did either).
Secure means also that the code is safe.
Publishing plugins in a repo for download without any sort of review (which was unfortunately the case in at least one plugin) is not going to be secure, no matter how many guidelines and guidelines will we have.

PS:
I totally agree that it is messy and frustrating that we have to go to a forum post to see what link to use to submit a plugin. This is however a problem of low community resources. There simply is no large team that can take care of all that at once.

Back on topic, as @pluginvulns points out, WP Plugin review is also not as secure as it could be. I am not speaking of employing 100 people to read the code, but if an automated tool even can find those holes, then (I think) it is a must to use that on CP, even to just make a difference to WP.

Unfortunately we do not have that large user base and need to distance ourselves from WP somehow to even give a reason as of why to use CP.
“Not having to use Blocks” is not enough, as we can see. Almost no one of the those who say “Blocks is [insert suboptimal opinion here]” are switching to CP. Why?
Because of 2 main reasons:

• basically no docs
• no plugins or themes whatsoever that make their live easier. Not everyone is a developer. There are thousands of users who do dislike blocks but prefer to use it, if they can pool from a large library of plugins and themes.

And yes, setting the bar high on our own repo is not going to increase that number of plugins or themes, but, at least, we could be able to say “these things are vetted by [insert vetting process here]”

I think this is, amongst the “no blocks” and “safe updates” a main leg for CP to stand on:
Security. Not a bunch of suboptimal plugins but few, vetted and well written ones

Of course this is just my opinion. Not trying to force it on anyone and I totally see the points in favour as well as those in counter.
I will try to join mentioned meeting, if just to listen, and share thoughts.

[azurecurve]

The very kind offer from @pluginvulns for use of their checker should, in my opinion, be taken up so we can do basic security checks. I was also going to take a look at the tool you mentioned the other day; I’ve got a VM setup with everything else needed (Git, PHP, VSCode and so on), I just need to work out the installation process for phpcs and wpcs (and the time to do it).

We also need the next directory update live and get the docs published so there is clarity for everyone on the submission method and guidelines.

[joyously]

This was posted in WP Slack a few minutes ago.

[wadestriebel]

For now, this draft version is up on the website. We will update it as required:

Submit Your Plugin | ClassicPress

[anon66243189]

Coming back there to ask if we can add something like an email sent to the dev as soon he/she submitted the plugin (like WP does) that confirms the plugin is now under review

Today I submitted a plugin which I can of course see (as under review) in my profile, but it might be just a “reassuring” thing to send an email as well, it can be automated, stating what will happen next + how long it’ll take and such things. BTW this also helps confirming the actual email of the DEV, because if the automated thing bounces, we don’t even need to review the plug

For devs who come after (in future) it might not be obvious what will happen after they submit, and thus again an email could be cool to have.

Thoughts?

[azurecurve]

I think there are plans to do this.

@wadestriebel can confirm?