Remove unrelated review rules from plugin review process

Plugins
1

These 2 rules are included in the Plugin Review Process:

  • Make sure the readme is clear and contains no contradictory information.
  • Upload plugin to a test environment with define(‘WP_DEBUG’, true); and activate; check for any errors on front and back ends.

I disagree this should be part of the review process.
The review process is not a “we test your plugin for you”. It is a “we make sure you do not submit bad stuff” process.
If you have a fatal error class not found on the first install, that is your problem as a DEV, and same for the readme information. (We will just make sure the readme does not smell like spam, for example, but other than that… it’s up to the dev)

Do we agree we can remove these 2 rules from Plugin Directory Review Process | ClassicPress Documentation?

3 Likes
2

I would agree with this. It’s just putting more workload on people like you who are trying to make this work. Anyway, more devs will probably use the plugin and sound the alarm if something is wrong.

1 Like
3

Yes, that was exactly my thought. It shouldn’t be our job to check the code as per functionality, that is the job of dev and users

OK, lets wait for some more feedback on it and I will remove it if all agree (or better: no one disagrees)

1 Like
4

I think the one about the readme is for information contradictory to CP’s philosophy, whatever that is. It could be non-GPL compatible or promoting hijacking websites for mining crypto or other things.
I think the whole point of the plugin directory is the subtle approval conferred upon the plugins in the list, and this extends to them being reviewed and yes, tested with an initial run-through. They should be worthy of being in the list, as the non-technical user doesn’t have much to go by to determine if a plugin is safe to use.
The reviews are for the helping the users; otherwise why have a list at all?

2 Likes
5

Exactly. And… of course plugins have to be reviewed. Run, see the code, see everything as much as possible. If a bad plugin goes to a user… it will be bad reputation for classic press. In any case, the plugins gallery is useless if not integrated with the admin zone. Priority: integrate the api asap. I can help, as i said in other topic.

6

I would not delete this. It’s the minimum that must be done to guarantee a minimum of quality.

1 Like
7

No, fatal errors become users’ problems the moment they install the plugin and get WSOD or errors popping up on their frontend/backend.

Listing plugins in the directory, as Joy pointed out, tells non-technical users that plugins are considered to be safe. Safety isn’t just about security, but also usability. If a user installs a plugin that bricks their website, it’s not safe.

Notices, warnings, etc. that do not break user’s website are fine, but plugins need to be checked for fatal errors to make sure they don’t break user’s website on install/activation.

One way to simplify this step. CP could maintain one instance for review process with WP_DEBUG enabled, and debug.log shown in the admin using PHP Error Log Viewer. We would just need a quick way to install a plugin from a Github URL. If reviewer has SSH access and uses WP-CLI, that’s a quick way to install it. With WP-CLI, wouldn’t even need Error Log viewer. This process could be automated, if not fully at least partially.

This probably should be the first step. Checking for fatal errors. If there are errors, dev is notified to fix the issue without any other review steps taken.

In any case, we should publish plugins that have fatal errors.

And on the other step “Make sure the readme is clear and contains no contradictory information.” This is very subjective. I would reword it and say “Make sure the readme is clear and contains no malicious information and/or spam”. And again, it’s subjective. A quick read of readme and if it sounds fine, that should be it. This is basically what you said, “We will just make sure the readme does not smell like spam”.

I’m all for simplifying the process, but we can’t publish unsafe plugins. People will expect the same level of safety from CP directory as they expect from WP repo.

8

I think the point about the readme should stay. As @joyously says, that covers things like incompatible licenses.

For various reasons, the debug thing should go, though. One such reason is that a later version might introduce something, but that would get missed because the original upload was fine.

Another reason is that, sometimes, it’s still worth having a plugin that causes such an issue. The most important plugin I use is Query Monitor. But for years it has caused a warning for Ajax issues. No-one cares because it doesn’t affect the functionality of the plugin, and it would be fairly hard to fix, so not worth time bothering while other improvements can be made.

2 Likes
9

I am conflicted about the review thing.

  1. We consider the dir a vetted list, this means testing the plugin and checking malicious code is all on us
  2. We consider it just a list, so we only check for malicious code once, and tell users “this is open software, you install and use at your own risk”.

Both have legal implications and need different rules.

  1. Needs a team doing just testing and checking while 2. can be automated

Both have good reasons to be selected.

At this stage vetting the list could however discourage people from submitting their plugins?

10

I think 2 is best addressed if/when a user complains about a plugin and gets no satisfactory response from the plugin developer.

2 Likes
11

A proper code review (cpcs) already spots all fatal errors. There’s no need to run the plugin.
In fact I spotted one yesterday doing this process

If someone wants to wp debug test plugins on an install they’re free to do so. One person alone can’t do that. There’s plugins they might hide a fatal anywhere in a setting you’d never even use as a tester.

The readme will be read and license is another point in the review as well as external contact / malicious code etc.
All this is already covered in the review process without making sure I’ve not accidentally added a contradicting detail to a readme or missed a semicolon in a php line by using wp debug and actually testing the plugin (the first is really up to the developer and his work, and the users reporting this glitch, the second is caught by code review)

We’re not the plugin developers team of free QC testers.

Ps;
The plugin with fatal error was in the directory since day one :rofl:
No one even noticed it. Now that plug-in’s gone, pending refactor / review, and that was spotted without actually „testing“ the plugin.

We can certainly have such vetting, but then first someone has to come forward and volunteer for it, and then also do it. Then we can add that point back as a „surplus“. I warn you: this is more than just enabling wp debug and „quickly install the plugin“ if done properly. It can mean setting up options, testing them, even checking if the fatals don’t suddenly appear due to conflicts etc. where would you set the limit? Imo the open source users are collaborators and not clients (this is one thing wp got right) and thus aren’t entitled to a (free) full blown QC done by the cp reviewers, but if anything they have to collaborate with the developer to make the software better.

3 Likes