Plugin Review Process | Some issues I think we should fix and details to clarify transparently

I have access to the Plugin Directory backend and I see ca. 14 plugins that are not reviewed/published in the directory.
I also see about 20 developers not published.

Since we have no log or common viewable history of communication accessible to the community (I believe there should, but I might be wrong), and I didn’t receive a clear reply to this issue in the relevant channels, and I am not in the review-email loop, I am not able to tell if those plugins and developers are pending review or have been rejected or wait for input or else.

I am however sure sure that at least one of these plugins (probably 2) has not been reviewed and the developer in question has not received a feedback since submission, whether their plugin has issues, or their profile has issues, or why the review takes longer than a reasonable amount of time (which I personally would set around a week, but I couldn’t find specifications on this in our guidelines, perhaps I missed it).

I believe this is suboptimal als not transparent enough and discouraging developers.

These are my proposals to fix this:

A) We should set a estimated time to review. We then should comply with that.
B) The developer should receive an email immediately after submission, stating we received the submission, with an estimate for review. This will also then allow that Developer to nag back on that email if the review takes longer, instead of wondering why, or where they could ask for further info (our plugin review email is not exactly easy to spot)
C) I believe it should be made public in the relevant forum here, when a new submission is made, what the status thereof is (use tags in the forum such as reviewing, rejected) and why it is rejected, if. That same for Developers Submissions.

If this is a bad idea, please explain why it isn’t adequate to do it and what is getting done to organise our review process so it has an actual log and discernible way of telling, now or in 100 years, to new or old review team members, why a certain plugin or developer isn’t published, and what the estimate of time is to get this done.

This additionally also would enable the community to actually test those plugins prior to release or even prior to review, and express issues on the thread that the review team, being humans only after all, might miss. This is not necessary of course, just a nice side effect it can have.


I know there’s an email thread but only reviewers have access to that, so it’s impossible for the community to inform themselves why perhaps a plugin or dev isn’t being published, to eventually directly help those folks when they ask about the issue. It is also probably not the easiest way to keep track of things as a reviewer itself.
I also know there’s a email that folks can use to inquire about the status of their plugins etc, but as reality shows, this isn’t used, and not really known.
Pretty sure that is because we do not send the initial “we got it” email, but even if we send that, the email is not really easy to find and/or might land in spam or trash folders for reasons.
It is buried in our DOC. Perhaps it should be on the Directory as a “contact” or so.

What cannot be or become the status quo

  • folks waiting for weeks or months and randomly comment here in the forum „never heard back from“ or else, while other plugins get published.
  • developers not being published (if there are reasons to reject 20 developers we probably have to do something as it is too many, and that would mean there is an issue with our process of onboarding developers.)
  • while above, other plugins being published, and developers + plugins that clearly do not follow our guidelines still being in the directory.

Further
There’s also been a discussion about completely suspend plugin submissions (in specific non public channels), and if that’s the decided case it has to be communicated here in public.
With reasons and (temporary) solutions.

5 Likes

I was writing about this and my plugin released and submitted 4 months ago:

I’ve not any interest in my plugin (that is defined unuseful by myself) to be published soon.

But if there is something wrong with my plugin I must know about.

And if there is something going bad in the directory the whole community must know about.

So thank @smileBeda for taking this issue public.

4 Likes

Overall I agree, but this is the part I disagree with. Specifically:

  1. Plugin review process is to ensure plugins comply with guidelines and are safe for the community to use. Community should not participate in this process until plugin is reviewed and published by the review team. Giving people access to a plugin before the review process defeats the purpose of the process.

  2. Learning from WordPress review process, reviewers should always remain anonymous. All emails should be from the review team without any identifying information. This is to protect revewer from harassment.

  3. Developers should definitely get a reason for rejection or suspension, and they can choose to bring it to the forums for a discussion. But the whole process doesn’t need to be public, this is especially true if it’s vulnerabilities or malware, etc.

Hopefully, we’ll get the directory integrated soon. I’m trying to get that process going again.

3 Likes

This are valid points.

To clarify - strictly speaking the plugins reviewed are already public at the moment of submission, prior to review. They must as a requirement be in GitHub, I believe.
As @Simone has shown, the plugin can even be announced here for example - without review - thus it’s not really making a difference if a plugin review status is shared here.

I agree reviewers should stay anonymous and security issues not disclosed publicly.
I’m not asking for a public review or names published of reviewers, but a public review log and process status and eventual reasons of denial, instead of obscuring the process.
And if a community member can test a plugin - I don’t think it hurts. It might actually generate more feedback. I could be wrong about that of course. The main point I want to touch is that right now, the process to me seems a black box.

3 Likes