Draft of the Plugin Directory Submission Guidelines

Off topic but perhaps useful for “review plugins” topic:

I’ve just finished setting up phpcs (with wpcs) on my machine and from now on the basic things won’t escape me anymore :v:

I can recommend this tool to anyone coding and with it installed, review team has a much easier task, as it recognizes many things (non escaped variables for example :face_with_hand_over_mouth: or Yoda conditions not respected) automatically, which increases safety a lot.

2 Likes

I don’t have experience reviewing plugins, but happy to be involved in this. Obviously, I’d need to recuse myself when my plugins are involved (it seems I also have a backlog of plugins to get submitted).

1 Like

When I was submitting plugins to WP, I had virtually no feedback from the WP plugin team. I know from the rewrites I did for ClassicPress that I had quite a few small security issues in plugins which were never picked up by the WP team.

1 Like

A decision needs to be taken on the level of reviewing which will/can be done and then after how it is presented in the directory.

I can take a look at the tool @anon66243189 mentioned as it does sound like it will be useful.

1 Like

Thank you, maybe let’s set up a community meeting for the week of July 5th and we can finalize this and get it moving forward :slight_smile:

Doodle Poll: https://doodle.com/poll/isg5fa6zfa8twwh3

1 Like

Preventing plugins into the directory because of a fear of security vulnerabilities is going to really hinder progress here.

Guidelines are there to guide developers and requirements for submission are there for developers to ensure their plugins meet certain minimum standards.

Good security practice is about clear education and experience.

The striving toward 100% security, or near to, is an effort in futility and will burn resources very quickly - and by resources I mean people. The enormous size of the codebase of some plugins makes it untenable to have reviews “by committee” - if you’re going to set the bar to be “as near to 100% safe as possible”, the testing for such a thing will always need to be automated.

Even then, you’re going to have security vulnerabilities. It’s the cost of doing business. A security vulnerability doesn’t “hurt” reputation in the long term really… it’s the response and handling of the vulnerability that really matters.

I feel in this discussion there’s a huge over-emphasis on security requirements and labelling, and choices/options etc. Developers vary in their competency, and so does their security handling within their code. There’s no getting around that. The WordPress plugin directory, while it has its flaws in some areas, gets the job done really well. It’s open to anyone to create and contribute and there’s no labels like “not reviewed” or “basic vulnerability reviewed”. Everyone accepts the risk that a plugin has undergone certain basic tests, or maybe not, and they get on with it.

If you want people to contribute and get involved, making the barrier to entry so high and adding complexity will defeat your purpose.

If you’re going to put requirements in place, you’ll need to make it very clear

  • clear documentation on what they are
  • how to meet them
  • what to do if things don’t pass
  • processes for updating the requirements and communicating updates.

A simple example of where none of this applies is the plugin submissions form/process.

I tried for the 2nd time to submit a plugin, using a dead link/form that’s been clearly non-functional for weeks. So I’m wasting my time doing that, then submitting forums requests, then digging around to see if I can find answers, the discovering that the guidelines to submissions are buried in the forums.

And it could have been avoided by having the dead link updated with a link to brief outline of the current guidelines and linking to the live submissions page.

My points is… if you’re going make rules and policies for developers to contribute, document and make it clear how things work before implementing them. Forums aren’t a substitute for documentation and clear guidelines.

7 Likes

We can provide some additional insight on that, which might help with decisions being made on in how this is handled with ClassicPress.

Currently, the team running the WordPress Plugin Directory is listed as having only four members. So while it is claimed that a manual review is done of each plugin, which includes checking the security, as a practical matter, that doesn’t seem like it could be happening. Other things we have seen point to them not happening in a lot of instances as well.

If the security review portion of that is happening, the results are not good, as two weeks ago we spotted two new plugins being added to their directory that contained serious vulnerabilities, which hackers would exploit. We spotted the possibility of those through an automated tool, which we have offered for years to provide them access to or offered to help them have similar capabilities, so there isn’t an excuse for that. The same week we spotted another recently added plugin that has serious vulnerability, which hackers would exploit, that a manual review should have caught, and our automated tool caught when the developer fixed the broken state of the code (which allowed the vulnerability to be exploited).

In other cases, when we have gone to check on new plugins with possible security issues, we couldn’t figure out if they were vulnerable because the plugin was too broken. That points the review not always happening and also an indication of the state you might find plugins being submitted to be in.

While having 100% security is nearly impossible, it wouldn’t be hard to provide better results than what WordPress does with plugins and what, unfortunately and troublingly, much of the security industry around WordPress seems to consider acceptable.

Even limited checks to see if basic security is in place, to check for the possibility of serious issues, or some combination of those two, would be a big improvement over what has WordPress done. As we mentioned in another topic, we can provide free access to our Plugin Security Checker, which flags the possibility of plenty of that. It requires some security knowledge to check over the results and certainly won’t catch everything, but it can help provide better results than WordPress is delivering. When we ran the plugins currently in your directory through that we found multiple plugins with vulnerabilities, including two WordPress plugins that had the vulnerabilities in them since their first versions, the version that was supposed to have been manually checked by the WordPress team.

4 Likes

I think this would be very useful; there is a meeting next week re: plugin submissions; I’ll make sure this offer gets a mention.

We nee dot find a line for security between @anon66243189 and @Paul where we encourage security, but not make the cost of entry too high. Not in quite the same way, but I think this is what WP is now doing with blocks and React; smaller/new developers won’t be able to access the marketplace in the same way as previously which will restrict competition and isn;t necessarily going to drive standards up.

1 Like

I absolutely agree that a bar cannot be set so high that no one can cross it.
And yet, this is CPs’ catchphrase:
Stable. Secure. Instantly Familiar.

Secure.
I think CP (should) tries to make a difference to WP here.
Secure does not only mean “an update wont break your site” (which no WP update ever really did either).
Secure means also that the code is safe.
Publishing plugins in a repo for download without any sort of review (which was unfortunately the case in at least one plugin) is not going to be secure, no matter how many guidelines and guidelines will we have.

PS:
I totally agree that it is messy and frustrating that we have to go to a forum post to see what link to use to submit a plugin. This is however a problem of low community resources. There simply is no large team that can take care of all that at once.

Back on topic, as @pluginvulns points out, WP Plugin review is also not as secure as it could be. I am not speaking of employing 100 people to read the code, but if an automated tool even can find those holes, then (I think) it is a must to use that on CP, even to just make a difference to WP.

Unfortunately we do not have that large user base and need to distance ourselves from WP somehow to even give a reason as of why to use CP.
“Not having to use Blocks” is not enough, as we can see. Almost no one of the those who say “Blocks is [insert suboptimal opinion here]” are switching to CP. Why?
Because of 2 main reasons:

  • basically no docs
  • no plugins or themes whatsoever that make their live easier. Not everyone is a developer. There are thousands of users who do dislike blocks but prefer to use it, if they can pool from a large library of plugins and themes.

And yes, setting the bar high on our own repo is not going to increase that number of plugins or themes, but, at least, we could be able to say “these things are vetted by [insert vetting process here]”

I think this is, amongst the “no blocks” and “safe updates” a main leg for CP to stand on:
Security. Not a bunch of suboptimal plugins but few, vetted and well written ones

Of course this is just my opinion. Not trying to force it on anyone and I totally see the points in favour as well as those in counter.
I will try to join mentioned meeting, if just to listen, and share thoughts.

The very kind offer from @pluginvulns for use of their checker should, in my opinion, be taken up so we can do basic security checks. I was also going to take a look at the tool you mentioned the other day; I’ve got a VM setup with everything else needed (Git, PHP, VSCode and so on), I just need to work out the installation process for phpcs and wpcs (and the time to do it).

We also need the next directory update live and get the docs published so there is clarity for everyone on the submission method and guidelines.

3 Likes

This was posted in WP Slack a few minutes ago.

3 Likes

For now, this draft version is up on the website. We will update it as required:

4 Likes

Coming back there to ask if we can add something like an email sent to the dev as soon he/she submitted the plugin (like WP does) that confirms the plugin is now under review

Today I submitted a plugin which I can of course see (as under review) in my profile, but it might be just a “reassuring” thing to send an email as well, it can be automated, stating what will happen next + how long it’ll take and such things. BTW this also helps confirming the actual email of the DEV, because if the automated thing bounces, we don’t even need to review the plug :wink:

For devs who come after (in future) it might not be obvious what will happen after they submit, and thus again an email could be cool to have.

Thoughts?

4 Likes

I think there are plans to do this.

@wadestriebel can confirm?

Yup!

1 Like

Seasoned WP dev here willing to volunteer as Plugin Review Team for CP Plugin Repo submissions. I am very enthused about ClassicPress and it’s growing community. Larry Judd - Codeable

8 Likes

Welcome to CP!

1 Like

Welcome to CP community. @james and @azurecurve would be the right guys to talk to about helping with plugin reviews.

2 Likes

@wadestriebel is the man to speak to about joining the team.

Edit to add: once you have access I can then run through the process I have been following.

2 Likes

I just dm’d you, thanks for helping out and then as @azurecurve mentioned it would be good to get you into the slack channel as well :+1:

3 Likes