Skip to main content

Generate a fully-working Go CRUD HTTP API with Ent

When we say that one of the core principles of Ent is "Schema as Code", we mean by that more than "Ent's DSL for defining entities and their edges is done using regular Go code". Ent's unique approach, compared to many other ORMs, is to express all of the logic related to an entity, as code, directly in the schema definition.

With Ent, developers can write all authorization logic (called "Privacy" within Ent), and all of the mutation side-effects (called "Hooks" within Ent) directly on the schema. Having everything in the same place can be very convenient, but its true power is revealed when paired with code generation.

If schemas are defined this way, it becomes possible to generate code for fully-working production-grade servers automatically. If we move the responsibility for authorization decisions and custom side effects from the RPC layer to the data layer, the implementation of the basic CRUD (Create, Read, Update and Delete) endpoints becomes generic to the extent that it can be machine-generated. This is exactly the idea behind the popular GraphQL and gRPC Ent extensions.

Today, we would like to present a new Ent extension named elk that can automatically generate fully-working, RESTful API endpoints from your Ent schemas. elk strives to automate all of the tedious work of setting up the basic CRUD endpoints for every entity you add to your graph, including logging, validation of the request body, eager loading relations and serializing, all while leaving reflection out of sight and maintaining type-safety.

Let’s get started!

Getting Started#

The final version of the code below can be found on GitHub.

Start by creating a new Go project:

mkdir elk-example
cd elk-example
go mod init elk-example

Invoke the ent code generator and create two schemas: User, Pet:

go run -mod=mod entgo.io/ent/cmd/ent init Pet User

Your project should now look like this:

.
├── ent
│ ├── generate.go
│ └── schema
│ ├── pet.go
│ └── user.go
├── go.mod
└── go.sum

Next, add the elk package to our project:

go get -u github.com/masseelch/elk

elk uses the Ent extension API to integrate with Ent’s code-generation. This requires that we use the entc (ent codegen) package as described here. Follow the next three steps to enable it and to configure Ent to work with the elk extension:

1. Create a new Go file named ent/entc.go and paste the following content:

// +build ignore
package main
import (
"log"
"entgo.io/ent/entc"
"entgo.io/ent/entc/gen"
"github.com/masseelch/elk"
)
func main() {
ex, err := elk.NewExtension()
if err != nil {
log.Fatalf("creating elk extension: %v", err)
}
err = entc.Generate("./schema", &gen.Config{}, entc.Extensions(ex))
if err != nil {
log.Fatalf("running ent codegen: %v", err)
}
}

2. Edit the ent/generate.go file to execute the ent/entc.go file:

package ent
//go:generate go run -mod=mod entc.go

3/. elk uses some external packages in its generated code. Currently, you have to get those packages manually once when setting up elk:

go get github.com/mailru/easyjson github.com/masseelch/render github.com/go-chi/chi/v5 go.uber.org/zap

With these steps complete, all is set up for using our elk-powered ent! To learn more about Ent, how to connect to different types of databases, run migrations or work with entities head over to the Setup Tutorial.

Generating HTTP CRUD Handlers with elk#

To generate the fully-working HTTP handlers we need first create an Ent schema definition. Open and edit ent/schema/pet.go:

package schema
import (
"entgo.io/ent"
"entgo.io/ent/schema/field"
)
// Pet holds the schema definition for the Pet entity.
type Pet struct {
ent.Schema
}
// Fields of the Pet.
func (Pet) Fields() []ent.Field {
return []ent.Field{
field.String("name"),
field.Int("age"),
}
}

We added two fields to our Pet entity: name and age. The ent.Schema just defines the fields of our entity. To generate runnable code from our schema, run:

go generate ./...

Observe that in addition to the files Ent would normally generate, another directory named ent/http was created. These files were generated by the elk extension and contain the code for the generated HTTP handlers. For example, here is some of the generated code for a read-operation on the Pet entity:

const (
PetCreate Routes = 1 << iota
PetRead
PetUpdate
PetDelete
PetList
PetRoutes = 1<<iota - 1
)
// PetHandler handles http crud operations on ent.Pet.
type PetHandler struct {
handler
client *ent.Client
log *zap.Logger
}
func NewPetHandler(c *ent.Client, l *zap.Logger) *PetHandler {
return &PetHandler{
client: c,
log: l.With(zap.String("handler", "PetHandler")),
}
}
// Read fetches the ent.Pet identified by a given url-parameter from the
// database and renders it to the client.
func (h *PetHandler) Read(w http.ResponseWriter, r *http.Request) {
l := h.log.With(zap.String("method", "Read"))
// ID is URL parameter.
id, err := strconv.Atoi(chi.URLParam(r, "id"))
if err != nil {
l.Error("error getting id from url parameter", zap.String("id", chi.URLParam(r, "id")), zap.Error(err))
render.BadRequest(w, r, "id must be an integer greater zero")
return
}
// Create the query to fetch the Pet
q := h.client.Pet.Query().Where(pet.ID(id))
e, err := q.Only(r.Context())
if err != nil {
switch {
case ent.IsNotFound(err):
msg := stripEntError(err)
l.Info(msg, zap.Error(err), zap.Int("id", id))
render.NotFound(w, r, msg)
case ent.IsNotSingular(err):
msg := stripEntError(err)
l.Error(msg, zap.Error(err), zap.Int("id", id))
render.BadRequest(w, r, msg)
default:
l.Error("could not read pet", zap.Error(err), zap.Int("id", id))
render.InternalServerError(w, r, nil)
}
return
}
l.Info("pet rendered", zap.Int("id", id))
easyjson.MarshalToHTTPResponseWriter(NewPet2657988899View(e), w)
}

Next, let’s see how to create an actual RESTful HTTP server that can manage your Pet entities. Create a file named main.go and add the following content:

package main
import (
"context"
"fmt"
"log"
"net/http"
"elk-example/ent"
elk "elk-example/ent/http"
"github.com/go-chi/chi/v5"
_ "github.com/mattn/go-sqlite3"
"go.uber.org/zap"
)
func main() {
// Create the ent client.
c, err := ent.Open("sqlite3", "./ent.db?_fk=1")
if err != nil {
log.Fatalf("failed opening connection to sqlite: %v", err)
}
defer c.Close()
// Run the auto migration tool.
if err := c.Schema.Create(context.Background()); err != nil {
log.Fatalf("failed creating schema resources: %v", err)
}
// Router and Logger.
r, l := chi.NewRouter(), zap.NewExample()
// Create the pet handler.
r.Route("/pets", func(r chi.Router) {
elk.NewPetHandler(c, l).Mount(r, elk.PetRoutes)
})
// Start listen to incoming requests.
fmt.Println("Server running")
defer fmt.Println("Server stopped")
if err := http.ListenAndServe(":8080", r); err != nil {
log.Fatal(err)
}
}

Next, start the server:

go run -mod=mod main.go

Congratulations! We now have a running server serving the Pets API. We could ask the server for a list of all pets in the database, but there are none yet. Let’s create one first:

curl -X 'POST' -H 'Content-Type: application/json' -d '{"name":"Kuro","age":3}' 'localhost:8080/pets'

You should get this response:

{
"age": 3,
"id": 1,
"name": "Kuro"
}

If you head over to the terminal where the server is running you can also see elks built in logging:

{
"level": "info",
"msg": "pet rendered",
"handler": "PetHandler",
"method": "Create",
"id": 1
}

elk uses zap for logging. To learn more about it, have a look at its documentation.

Relations#

To illustrate more of elks features, let’s extend our graph. Edit ent/schema/user.go and ent/schema/pet.go:

ent/schema/pet.go
// Edges of the Pet.
func (Pet) Edges() []ent.Edge {
return []ent.Edge{
edge.From("owner", User.Type).
Ref("pets").
Unique(),
}
}
ent/schema/user.go
package schema
import (
"entgo.io/ent"
"entgo.io/ent/schema/edge"
"entgo.io/ent/schema/field"
)
// User holds the schema definition for the User entity.
type User struct {
ent.Schema
}
// Fields of the User.
func (User) Fields() []ent.Field {
return []ent.Field{
field.String("name"),
field.Int("age"),
}
}
// Edges of the User.
func (User) Edges() []ent.Edge {
return []ent.Edge{
edge.To("pets", Pet.Type),
}
}

We have now created a One-To-Many relation between the Pet and User schemas: A pet belongs to a user, and a user can have multiple pets.

Rerun the code generator:

go generate ./...

Do not forget to register the UserHandler on our router. Just add the following lines to main.go:

[...]
r.Route("/pets", func(r chi.Router) {
elk.NewPetHandler(c, l, v).Mount(r, elk.PetRoutes)
})
+ // Create the user handler.
+ r.Route("/users", func(r chi.Router) {
+ elk.NewUserHandler(c, l, v).Mount(r, elk.UserRoutes)
+ })
// Start listen to incoming requests.
fmt.Println("Server running")
[...]

After restarting the server we can create a User that owns the previously created Pet named Kuro:

curl -X 'POST' -H 'Content-Type: application/json' -d '{"name":"Elk","age":30,"owner":1}' 'localhost:8080/users'

The server returns the following response:

{
"age": 30,
"edges": {},
"id": 1,
"name": "Elk"
}

From the output we can see that the user has been created, but the edges are empty. elk does not include edges in its output by default. You can configure elk to render edges using a feature called "serialization groups". Annotate your schemas with the elk.SchemaAnnotation and elk.Annotation structs. Edit ent/schema/user.go and add those:

// Edges of the User.
func (User) Edges() []ent.Edge {
return []ent.Edge{
edge.To("pets", Pet.Type).
Annotations(elk.Groups("user")),
}
}
// Annotations of the User.
func (User) Annotations() []schema.Annotation {
return []schema.Annotation{elk.ReadGroups("user")}
}

The elk.Annotations added to the fields and edges tell elk to eager-load them and add them to the payload if the " user" group is requested. The elk.SchemaAnnotation is used to make the read-operation of the UserHandler request " user". Note, that any fields that do not have a serialization group attached are included by default. Edges, however, are excluded, unless configured otherwise.

Next, let’s regenerate the code once again, and restart the server. You should now see the pets of a user rendered if you read a resource:

curl 'localhost:8080/users/1'
{
"age": 30,
"edges": {
"pets": [
{
"id": 1,
"name": "Kuro",
"age": 3,
"edges": {}
}
]
},
"id": 1,
"name": "Elk"
}

Request validation#

Our current schemas allow to set a negative age for pets or users and we can create pets without an owner (as we did with Kuro). Ent has built-in support for basic validation. In some cases you may want to validate requests made against your API before passing their payload to Ent. elk uses this package to define validation rules and validate data. We can create separate validation rules for Create and Update operations using elk.Annotation. In our example, let’s assume that we want our Pet schema to only allow ages greater than zero and to disallow creating a pet without an owner. Edit ent/schema/pet.go:

// Fields of the Pet.
func (Pet) Fields() []ent.Field {
return []ent.Field{
field.String("name"),
field.Int("age").
Positive().
Annotations(
elk.CreateValidation("required,gt=0"),
elk.UpdateValidation("gt=0"),
),
}
}
// Edges of the Pet.
func (Pet) Edges() []ent.Edge {
return []ent.Edge{
edge.From("owner", User.Type).
Ref("pets").
Unique().
Required().
Annotations(elk.Validation("required")),
}
}

Next, regenerate the code and restart the server. To test our new validation rules, let’s try to create a pet with invalid age and without an owner:

curl -X 'POST' -H 'Content-Type: application/json' -d '{"name":"Bob","age":-2}' 'localhost:8080/pets'

elk returns a detailed response that includes information about which validations failed:

{
"code": 400,
"status": "Bad Request",
"errors": {
"Age": "This value failed validation on 'gt:0'.",
"Owner": "This value is required."
}
}

Note the uppercase field names. The validator package uses the structs field name to generate its validation errors, but you can simply override this, as stated in the example .

If you do not define any validation rules, elk will not include the validation-code in its generated output. elks` request validation is especially useful if you'd wanted to do cross-field-validation.

Upcoming Features#

We hope you agree that elk has some useful features already, but there are still many exciting things to come. The next version of elk will include::

  • Fully working flutter frontend to administrate your nodes
  • Integration of Ent’s validation in the current request validator
  • More transport formats (currently only JSON)

Conclusion#

This post has shown just a small part of what elk can do. To see some more examples of what you can do with it, head over to the project’s README on GitHub. I hope that with elk-powered Ent, you and your fellow developers can automate some repetitive tasks that go into building RESTful APIs and focus on more meaningful work.

elk is in an early stage of development, we welcome any suggestion or feedback and if you are willing to help we'd be very glad. The GitHub Issues is a wonderful place for you to reach out for help, feedback, suggestions and contribution.

About the Author#

MasseElch is a software engineer from the windy, flat, north of Germany. When not hiking with his dog Kuro (who has his own Instagram channel 😱) or playing hide-and-seek with his son, he drinks coffee and enjoys coding.