A better meta API for WordPress

One of the things that we have being discussing for a very long time is extending WordPress with a comment meta api or even the idea of a generic meta api for WordPress and indeed this is something we are discussing at the moment and I thought I would jot down some thoughts on what I would like to see from an API point of view.  Over the weekend at WordCamp UK we also heard about situations where some people are already adding comment meta tables for plugin usage and so the demand is definitely there.

I don’t really care how the data is stored, be it single table or multi table, all I care about is having a good stable API for plugins and the core to work with.  If the API is good and well thought out they don’t need to care about the table structure and we can always change it later.

Therefore I thought I would summarise the features I would like to see in a generic meta api:

  • Flexibility to easily create a different kind of meta without having to care about adding tables yourself.
  • Ability to store anything in a meta value – i.e. The same kind of functionality we have we options.
  • Ability to get things back based on key ranges or operations – i.e. Getting all cron meta values where the meta key (which would be a timestamp) is before a particular time.

So I am thinking of an api like this:

/*
* Register a new meta type.
* If we have a table per meta this will create the table for you if required
*/
register_meta_type('cron');

/*
* Returns the meta value for a particular key
*/
get_meta_value('cron', $key);

/*
* Sets the meta value for a particular key
*/
set_meta_value('cron', $key, $value);

/*
* Returns the meta values for a particular key based search
*/
get_meta_values('cron', $search_value, $search_type);

/*
* Deletes the meta value for a particular key
*/
delete_meta_value('cron', $key);

I would envisage us enabling the use of the new api with wrapper functions for different meta types as required. These wrapper functions would only be included if required, for example we could create a comment_meta wrapper api around these generic meta api functions which would only be available if a plugin / theme called enable_comment_meta_api()

10 Responses to “A better meta API for WordPress”

  1. Beau lebens says:

    If that was used to replace something like postmeta, then wouldn’t you want some sort of meta_type_id in there as well? e.g. so that you could get the key=’post_custom_field_name’, that belongs to a specific post? You could just add it on like this:

    get_meta_value( ‘post’, ‘post_custom_field_name’, 123 );

    putting the meta_type_id last would allow you to make it optional easily, so that for something like your example (cron) where you might not have a numeric id that makes sense, it could be automatically handled (or completely ignored) internally. That would allow you to index the tables on that value, speeding up lookups a lot (since if we stick all meta in one big table, it’s going to get pretty big, pretty quickly).

    • westi says:

      Very true, something I had missed out on in this is an associated object_id for the meta entries when they are directly associated like post, comment and attachment meta would be.

  2. westi says:

    If we design the API well enough the consumers of the api don’t need to know about the table structure underneath.

    If people need different ways of accessing the data then we should add API calls for it – plugins shouldn’t need to use raw SQL queries to achieve things as then we lose the benefits that caching can provide.

    If we have multi-table we are likely to end up with a lot of similar looking tables, if we have single table we will need an extra column for the type of meta – the only choice between the two is going to be performance.

    In both cases we should be able to write an api which means that plugins don’t do raw SQL queries – that way we could make it possible for people to use either approach depending on there scaling needs. But that’s all implementation detail ;-)

    • scribu says:

      As a counter-example, I have a plugin that does some funky WHERE clauses to retrieve posts based on meta values. There’s no way of achieving that without raw SQL.

      I assume that other plugins that would be relying on comment meta, for example, would need similar functionality.

    • westi says:

      @scribu: Is what the plugin does really not possible we the post meta key / value stuff built into WP_Query e.g. – https://core.trac.wordpress.org/ticket/7431 and https://core.trac.wordpress.org/browser/trunk/wp-includes/query.php#L2140

      Yes. There will always be cases which are the exception that breaks the rule.

      We should strive to provide a good enough api so that plugins don’t have to use direct SQL queries.

      We try very hard to maintain apis we are not interested in maintaining table structures just to not break plugins.

  3. scribu says:

    @scribu: Is what the plugin does really not possible we the post meta key / value stuff built into WP_Query e.g. – https://core.trac.wordpress.org/ticket/7431 and https://core.trac.wordpress.org/browser/trunk/wp-includes/query.php#L2140

    No, because it has to check for more than one key-value pair.

    We should strive to provide a good enough api so that plugins don’t have to use direct SQL queries.

    Completely agree.

    We try very hard to maintain apis we are not interested in maintaining table structures just to not break plugins.

    I know, but why break plugins unnecessarily? :)

    Anyway, I’m glad we agree that plugins shouldn’t have to deal with creating meta tables, no matter the type.

  4. @westi: “If the API is good and well thought out they don’t need to care about the table structure and we can always change it later.”

    While in general I agree with you, strict adherence to your words could paint us into a nasty corner. Let me use an example: The API could be implemented to store all meta for a post as a serialized array in the wp_posts table. Doing so would adhere to your “table structure doesn’t matter” ideal but would be disastrous for enabling additional functionality. If meta was implemented in that manner it would be effectively impossible to implement a performant filter on a list of posts for a given meta key/value pair because every record would have to be tested via PHP instead of pushing to SQL in a WHERE clause.

    This is the exact problem I had with NexGen Gallery in that it used serialized arrays instead of a proper linking table. In order to implement functionality required by my client I had to modify the code for that plugin.

    So yes, ideally a good API should be all that matters but in reality the table structure does matter.

    • westi says:

      Yes we need to get the storage correct.

      My point was the most important thing is the API – if we get that wrong it’s much harder to change.

      Making incremental changes to the db structure to optimise it when issues arise is much easier that fixing a broken API :-)

  5. scribu says:

    I’ve submitted a basic patch to #10487.

    I would like to know what you had in mind with $search_type in

    get_meta_values(‘cron’, $search_value, $search_type);

    • westi says:

      The idea here was to make it possible to query the meta data in appropriate ways.

      I don’t have any specific examples – but the aim would be to ensure that plugins can avoid direct queries.



d
go to dashboard
l
go to login
h
show/hide help
e
edit post/page
r
comment on post/page
m
go to moderate comments
esc
cancel
%d bloggers like this: