The LLM content seems distracting and unhelpful to me.

In an example[1], three paragraphs start with "Oh boy" and "Oh dear". There are also "whopping 272 invalid schema examples" (twice), "whopping 334 rate limit responses", and a recommendation to "add rate limiting to a whopping 173 operations".

The prompts[2] contain "You like chatting in a playful, and a somewhat snarky manner.", "Keep the tone casual and playful, and a bit snarky.", so this seems partly intentional.

But even without the snark the AI summary looks unhelpful to me and I'd prefer a tool without LLM. So, Spectral[3] I guess.

[1]: https://ratemyopenapi.com/report/31bae2fb-bda1-471b-8c1d-142...

[2]: https://github.com/zuplo/rate-my-openapi/blob/0fcef3702592d8...

[3]: https://github.com/stoplightio/spectral

This seems to be a fancy App/API surrounding Stoplight's Spectral library (https://docs.stoplight.io/docs/spectral/674b27b261c3c-overvi...), which is heavily relied upon (https://github.com/zuplo/rate-my-openapi/blob/main/apps/api/...) and not called out anywhere in the documentation.

When I originally looked at it, I assumed that there was going to be some collaboration or other form of discussion tool to help others navigate your spec file and provide feedback. As it is, it doesn't feel like its adding the much value.

We'll update the docs to make our dependency on vacuum / spectral clearer - fair feedback. We do more than this though, we also use LLMs to help us identify other issues and recommend fixes.

RateMyOpenAPI is really designed as a quick litmus test to help people get their OpenAPI doc (and thus their documentation) in a good shape quickly, vs the hassle of setting up of individual rules.

People find it quite helpful to quickly build a todolist of things to do with their OpenAPI doc to get it into a much better state.

I had a brief look at the tool. It's a pity but it's not a tool, it's a service with unclear ToS. The CLI tool is a wrapper which sends openapi definition to the service. That's a big NO.

In general I get a feeling that it's just a small feature who wants to become a paid product.

Co-founder of Zuplo here - we actually created the service to help people get their docs into better shape (we use your OpenAPI to power the dev portal in Zuplo). We find that if folks have a high ratemyopenapi score their docs tend to be pretty good. We have no plans to charge for this service.

Thanks for the feedback, appreciate you taking the time to try it out and comment.

Most OpenAPI specs I work with are effectively under NDA, so submitting them to an external service is not an option, paid or not. There doesn’t seem to be any good reason why this couldn’t be a standalone tool.

I love such low value many words responses! If you have no plans to charge for the service but still provide it as a service instead of making it an offline tool, then it's even bigger NO.

We, some users, find that when there is no price for the service then we're paying with our data. It may be sold or used in unintended ways. It should be mentioned somewhere in ToS what you're going to do with the data, right? Oh wait, there is no ToS...

I'm sorry, but this comment is asinine.

1. Its fine to implement this as a web service. It makes it much more flexible to implement in multiple locations. (CLI, website, other backend)

2. Its something someone spent time making, that could help you, that theyre giving away at no cost. Future aside thats a nice gesture that should be celebrated.

3. OpenAPI specs on average are not precious, not very valuable to other parties.

4. If yours is, and contains super secret proprietary information you can't afford to leak, thats ok, you do not have to use this.

> 1. Its fine to implement this as a web service. It makes it much more flexible to implement in multiple locations. (CLI, website, other backend)

Rating an API specification using a set of rules based on opensource linter is a trivial task which can be easily implemented in such a way so it can be used both offline and online. Relying on a service adds additional latency.

> 2. Its something someone spent time making, that could help you, that theyre giving away at no cost. Future aside thats a nice gesture that should be celebrated.

Feedback was asked, feedback was received. Running the service which does server side processing costs money. There is no free lunch. They are providing the CLI so why not implement it directly there to save the costs?

> 3. OpenAPI specs on average are not precious, not very valuable to other parties. >4. If yours is, and contains super secret proprietary information you can't afford to leak, thats ok, you do not have to use this.

That's why there should be Terms of Service - they need to say how they are going to use/process my data on their servers. I know how much value my data has and I want to know what I am signing for.

Can you say some about how Rate My OpenAPI compares to Spectral?

Spectral is a JSON/YAML linter with custom rulesets, with out of the box support of OpenAPI and AsyncAPI.

I use it Spectral for OpenAPI feature analysis, much as described here: https://github.com/stoplightio/spectral-documentation

Rate My OpenAPI uses Spectral under the hood for a lot of the checks. This is meant to be a quick (and fun) service to help people improve their API docs. Doing full linting with Spectral is complementary - and more than this tool does.

Happy to see this. OpenAPI has been frustrating to me for how "aspirational" much of it is. Reminds me of the WSDL days when you were promised that writing the WSDL would help get cross vendor clients. Reality was more that the tools that worked in the MS world did not work in the Sun world. Heaven help you if you tried to get something working in both.

What I was seeing in OpenAPI docs is that everyone seemed stuck on Swagger for most interop. And if you really wanted generated clients/servers, you were likely using a framework where the fact that it also generated a swagger doc was a bit of a side benefit that didn't actually matter.

What about a comparison to vacuum (https://quobix.com/vacuum/) - the supposed killer of Spectral?

I've done this a little less quantitative using LLMs. Ask it to roast your API spec, give it a score. Could be expanded by more elaborate few-shot prompting: https://x.com/MartinHN/status/1775530951138746715

Hug of death? I'm getting a '500 Internal Server Error' when I press "open the report." Also, it's unclear how to obtain the api key from the README.

btw. maybe it might be better if this was integrated into the linter of your framework that builds the OpenAPI specification?

> btw. maybe it might be better if this was integrated into the linter of your framework that builds the OpenAPI specification?

Zuplo is has an Api Manager product "designed for developers". How would they be able to market and sell their product if it was just a CLI tool? Or gather developers' contact information for sending ads? This service is a good honeypot.

Sorry for being salty but it smells a bit fishy here - there are a lot of representatives in comments thanking for feedback but avoiding all interesting questions

OpenAPI is just pointless abstraction like GraphQL

It's a documentation spec that has a lot of tooling around it. I disagree that it is either pointless or an abstraction.

Back in the day, the best feature of SOAP API's was that they were documented in a machine readable fashion. Point your IDE at the WSDL endpoint and it would autogenerate an API client for you. OpenAPI and the various tooling around it finally offer us something similarly convenient for REST API's.

OpenAPI is a description, not an abstraction.