Project Structure - Core

Core is the main server for Openlane and where the majority of the backend code and business logic is located for the API.

cmd

---

server

Located in cmd/ this is a cobra cli that will start up the main core server. The Taskfile included in the repo provides the common commands but under the hood the serve command is what is being used

go run main.go serve --debug --pretty

cli

Located in cmd/cli/cmd this directory includes all the CRUD operations for the openlane cli. As these follow a pretty similar pattern throughout all objects, there is a template that allows these files to be generated and only requires the input and output fields to be updated.

info

If you are developing the cli or related functionality, instead of using the brew installed cli, you can use:

go run cmd/cli/main.go [command] [subcommand] <flags>

config

---

Located in config/ this directory contains the generated config and example settings powered by koanf.

If changes are made to the config structure or dependent configs, you'll need to regenerate the config:

task config:generate

info

The first time you setup the server locally, you'll need to copy the config/config-dev.example.yaml into config/.config.yaml. This file is in the .gitignore to prevent accidentally committing secrets

internal

---

Located in internal/ent, this is the meat of the repo, if you are looking for something, it's probably in this directory somewhere. Everything in the internal directory is intended to only be used by packages within this repository, outside usage will be blocked by the go-compiler.

ent

---

Located in internal/ent this directory contains all the code related to the schemas and the interactions with the database

structure

├── internal
│   ├── ent
│   │   ├── generated
│   │   ├── hooks
│   │   ├── interceptors
│   │   ├── mixin
│   │   ├── privacy
│   │   ├── schema
│   │   ├── templates
│   │   └── validator

schema

This include all schemas for the database. When generating a new schema, refer to the new schema documentation.

When generating a new schema, a template is used that includes the basic structure of any new schema used in Openlane

    package schema

    import (
        "entgo.io/contrib/entgql"
        "entgo.io/ent"
        "entgo.io/ent/schema"

        "github.com/theopenlane/core/internal/ent/mixin"
    )

    // Meow holds the schema definition for the Meow entity
    type Meow struct {
        ent.Schema
    }

    // Fields of the Meow
    func (Meow) Fields() []ent.Field {
        return []ent.Field{
            // Fields go here
        }
    }

    // Mixin of the Meow
    func (Meow) Mixin() []ent.Mixin {
        return []ent.Mixin{
            emixin.AuditMixin{},
            emixin.IDMixin{},
            mixin.SoftDeleteMixin{},
            emixin.TagMixin{},
        }
    }

    // Edges of the Meow
    func (Meow) Edges() []ent.Edge {
        return []ent.Edge{
            // Edges go here
        }
    }

    // Indexes of the Meow
    func (Meow) Indexes() []ent.Index {
        return []ent.Index{}
    }

    // Annotations of the Meow
    func (Meow) Annotations() []schema.Annotation {
        return []schema.Annotation{
            entgql.QueryField(),
            entgql.RelayConnection(),
            entgql.Mutations(entgql.MutationCreate(), (entgql.MutationUpdate())),

            // the above annotations create all the graphQL goodness; if you need the schema only and not the endpoints, use the below annotation instead
            // if you do not need the graphql bits, also be certain to add an exclusion to scripts/files_to_skip.txt
            // entgql.Skip(entgql.SkipAll),

            // the below annotation adds the entfga policy that will check access to the entity
            // remove this annotation (or replace with another policy) if you want checks to be defined
            // by another object
            entfga.SelfAccessChecks(),
        }
    }

    // Hooks of the Meow
    func (Meow) Hooks() []ent.Hook {
        return []ent.Hook{}
    }

    // Interceptors of the Meow
    func (Meow) Interceptors() []ent.Interceptor {
        return []ent.Interceptor{}
    }

    // Policy of the Meow
    func (Meow) Policy() ent.Policy {
        // add the new policy here, the default post-policy is to deny all
        // so you need to ensure there are rules in place to allow the actions you want
        return policy.NewPolicy(
            policy.WithQueryRules(
                // add query rules here, the below is the recommended default
                entfga.CheckReadAccess[*generated.MeowQuery](),
            ),
            policy.WithMutationRules(
                // add mutation rules here, the below is the recommended default
                policy.CheckCreateAccess(),
                entfga.CheckEditAccess[*generated.MeowMutation](),
            ),
        )
    }

hooks

This contains all hooks written to change the behavior of a mutation. Think of a hook as a middleware that can occur before or after the mutation is executed.

// HookExample is a hook that is used as an example that only happens on `Create` operations
func HookExample() ent.Hook {
    return hook.On(func(next ent.Mutator) ent.Mutator {
        return hook.ExampleFunc(func(ctx context.Context, m *generated.ExampleMutation) (generated.Value, error) {
            // code here will occur before the example mutation is executed

            retVal, err := next.Mutate(ctx, m)
            if err != nil {
                return nil, err
            }

            // code here will occur after the example mutation is executed

            return retVal, err
        })
    }, ent.OpCreate) // only do the thing on create operations, this can be omitted completely or include multiple operations separated by `|`
}

For more information, refer to the upstream docs

interceptors

This contains all interceptors written to change the behavior of a query. These are similar to hooks, but instead of acting on a mutation, it acts on a query.

• Traverse functions occur before the query is executed
• Interceptor functions occur after the query is executed

We commonly use these to filter data the user has access to, or log information such as query timing, for example:

    func QueryLogger() ent.InterceptFunc {
        return func(next ent.Querier) ent.Querier {
            return ent.QuerierFunc(func(ctx context.Context, query generated.Query) (ent.Value, error) {
                q, err := intercept.NewQuery(query)
                if err != nil {
                    return nil, err
                }

                start := time.Now()
                defer func() {
                    log.Info().
                        Str("duration", time.Since(start).String()).
                        Str("schema", q.Type()).
                        Msg("query duration")
                }()

                return next.Query(ctx, query)
            })
        }
    }

For more information, refer to the upstream docs

privacy

This includes privacy policies, this is used in conjunction with our FGA implementation

For more information, refer to the upstream docs

mixin

Mixins are common fields and functions that are used on multiple schemas, for example, the SoftDeleteMixin is used on all schemas as we want to use soft-deletes across all our database tables.

note

not all mixins are here. Due to import cycles, some are located within the schema directory. Others, because they are not unique to this repository are located within the entx repository

generated

All code generated by entc is in this directory and should not be edited manually as changes will be overwritten on the next run of task generate.

graphapi

---

Located in internal/graphapi this directory contains all the code related to graphapi resolvers,

All graphapi resolvers require authentication, this is handled by the auth middleware and requires no updates to the resolvers themselves.

structure

├── internal
│   ├── graphapi
│   │   ├── clientschema
│   │   ├── generate
│   │   ├── generated
│   │   ├── model
│   │   ├── query
│   │   ├── schema
│   │   └── testdata

resolvers

The *.resolvers.go files contain the business logic of the resolvers. These are generated based on the schema and a template, but the generated should be reviewed and updated as needed depending on the use case.

All of the list functions are included in ent.resolvers.go, whereas the CRUD functions are in a file per schema. For example, the group schema resolvers will be located in group.resolvers.go.

query

The internal/graphapi/query directory contains queries and mutations used to generate the openlaneclient using gqlgenc

info

Basic queries are generated as part of the gqlgen process, however, these only include direct fields and no edges. These are not regenerated after each run to preserve manual changes.

schema

The internal/graphapi/schema directory contains the generated graphql schemas.

You can add manually created schemas here for additional functionality. As an example the programextended.graphql contains extended schemas such as:

    extend input UpdateProgramInput {
        addProgramMembers: [CreateProgramMembershipInput!]
    }

    extend input ProgramMembershipWhereInput {
        programID: String
        userID: String
    }

info

Schemas are generated as part of the gqlgen process. These are not regenerated after each run to preserve manual changes with the exception of ent.graphql which is fully generated.

generated

All the generated functions are included in the internal/graphapi/generated directory, with a file per schema.

The generated models are included in the internal/graphapi/model/gen_model.go.

The files in both of these directories are not intended for manual changes, and instead should only be updated the the generation scripts.

The generate directory contains the config and generation scripts for the graphapi related generation (gqlgen and gqlgenc)

httpserve

---

server

Located in internal/httpserve this directory contains the main http server and configuration as well as the REST api handlers and routes. Although the majority of the openlane API is a graphapi, there are several REST routes such as login, password-reset, etc.

├── internal
│   ├── httpserve
│   │   ├── authmanager
│   │   ├── config
│   │   ├── handlers
│   │   ├── route
│   │   ├── server
│   │   └── serveropts

handlers

Located in internal/httpserve/handlers this directory contains the REST api handlers. The general use-case should not require a REST endpoint, however, there are several cases where REST handlers have been implemented such as for login. Handlers should all follow the same pattern of created a Handler and a BindHandler, for openAPI specs.

func (h *Handler) ExampleHandler(ctx echo.Context) error {
    // bind the input
    var in models.ExampleRequest
    if err := ctx.Bind(&in); err != nil {
        return h.InvalidInput(ctx, err)
    }

    // validate the input
    if err := in.Validate(); err != nil {
        return h.InvalidInput(ctx, err)
    }

    //
    // ... do some stuff ...
    //


    // return the response
    out := models.ExampleReply{
        Reply:    rout.Reply{Success: true},
        Message:  "success",
    }

    return h.Success(ctx, out)
}

// BindExampleHandler binds the example request to the OpenAPI schema
func (h *Handler) BindExampleHandler() *openapi3.Operation {
    example := openapi3.NewOperation()
    example.Description = "Example is ... "
    example.Tags = []string{"example"}
    example.OperationID = "ExampleHandler"

    h.AddRequestBody("ExampleRequest", models.ExampleExampleSuccessRequest, example)
    h.AddResponse("ExampleReply", "success", models.ExampleExampleSuccessResponse, example, http.StatusOK)
    example.AddResponse(http.StatusInternalServerError, internalServerError())
    example.AddResponse(http.StatusBadRequest, badRequest())

    return example
}

routes

Located in internal/httpserve/route this directory contains all the http routes for the server. All handlers must be registered routes with the echo server. Once a handler is created, a route should be added here. These all follow a predictable pattern

func registerExampleHandler(router *Router) (err error) {
    path := "/example"
    method := http.MethodPost
    name := "Example"

    route := echo.Route{
        Name:        name,
        Method:      method,
        Path:        path,
        Middlewares: mw,
        Handler: func(c echo.Context) error {
            return router.Handler.ExampleHandler(c)
        },
    }

    op := router.Handler.BindExampleHandler()

    if err := router.Addv1Route(path, method, op, route); err != nil {
        return err
    }

    return nil
}

info

The default middleware (mw) is an unauthenticated route. If the REST endpoint should require authentication authMW should be used instead.

server opts

This contains server options for the echo server, including With options for setting up different settings.

pkg

middleware

Located in pkg/middleware this contains many commonly used middleware used by the core server.

note

These may move to internal if they are dependent on the schema, or to another repo at some point in the future.

├── pkg
│   ├── middleware
│   │   ├── auth
│   │   ├── cachecontrol
│   │   ├── cors
│   │   ├── debug
│   │   ├── mime
│   │   ├── ratelimit
│   │   ├── ratelimiter
│   │   ├── redirect
│   │   ├── secure
│   │   └── transaction

models

Located in pkg/models this includes all the model information for the REST api input and output. This is used to generate our openAPI specs.

All REST endpoints should include an entry in the models package with:

• Request - the fields, noting omitempty if not required, that should be included with the POST request
• Reply - the fields included in a response from the handler
• Validate Request function - validation function for the Request
• ExampleSuccessRequest - example request containing valid values for the fields
• ExampleSuccessReply - example response that would be returned on a successful request

openlaneclient

Located in pkg/openlaneclient this includes the generated golang API client used to interact with the API. The queries added to internal/graphapi/query are used as input to gqlgenc for the graphclient. The restclient is manually maintained.