TypeORM how to seed database
For those who are using TypeORM with Nest.js, here is a solution to perform your seeding programatically, from within your code.
Rough idea:
- We create a dedicated "seeding module" containing a "seeding middleware" that is responsible for conducting the seeding and ensuring that all seeding is done before any request is answered.
- For any request that arrives, the seeding middleware intercepts it and postpones it until it is confirmed that seeding is done.
- If the db has been seeded, the "seeding middleware" passes the request to the next middleware.
- To speed things up, the "seeding middleware" keeps a "seeding complete" flag as state in memory to avoid any further db-checks after the seeding has occurred.
Implementation:
For this to work, first create a module that registers a middleware that listens to all incoming requests:
// file: src/seeding/SeedingModule.ts
@Module({})
export class SeedingModule implements NestModule {
configure(consumer: MiddlewareConsumer) {
consumer
.apply(SeedingMiddleware)
.forRoutes('*')
}
}
Now create the middleware:
// file: src/seeding/SeedingMiddleware.ts
import { Injectable, NestMiddleware } from '@nestjs/common';
import { Request, Response } from 'express';
import { EntityManager } from 'typeorm';
import { SeedingLogEntry } from './entities/SeedingLogEntry.entity';
@Injectable()
export class SeedingMiddleware implements NestMiddleware {
// to avoid roundtrips to db we store the info about whether
// the seeding has been completed as boolean flag in the middleware
// we use a promise to avoid concurrency cases. Concurrency cases may
// occur if other requests also trigger a seeding while it has already
// been started by the first request. The promise can be used by other
// requests to wait for the seeding to finish.
private isSeedingComplete: Promise<boolean>;
constructor(
private readonly entityManager: EntityManager,
) {}
async use(req: Request, res: Response, next: Function) {
if (await this.isSeedingComplete) {
// seeding has already taken place,
// we can short-circuit to the next middleware
return next();
}
this.isSeedingComplete = (async () => {
// for example you start with an initial seeding entry called 'initial-seeding'
// on 2019-06-27. if 'initial-seeding' already exists in db, then this
// part is skipped
if (!await this.entityManager.findOne(SeedingLogEntry, { id: 'initial-seeding' })) {
await this.entityManager.transaction(async transactionalEntityManager => {
await transactionalEntityManager.save(User, initialUsers);
await transactionalEntityManager.save(Role, initialRoles);
// persist in db that 'initial-seeding' is complete
await transactionalEntityManager.save(new SeedingLogEntry('initial-seeding'));
});
}
// now a month later on 2019-07-25 you add another seeding
// entry called 'another-seeding-round' since you want to initialize
// entities that you just created a month later
// since 'initial-seeding' already exists it is skipped but 'another-seeding-round'
// will be executed now.
if (!await this.entityManager.findOne(SeedingLogEntry, { id: 'another-seeding-round' })) {
await this.entityManager.transaction(async transactionalEntityManager => {
await transactionalEntityManager.save(MyNewEntity, initalSeedingForNewEntity);
// persist in db that 'another-seeding-round' is complete
await transactionalEntityManager.save(new SeedingLogEntry('another-seeding-round'));
});
}
return true;
})();
await this.isSeedingComplete;
next();
}
}
Finally here is the entity that we use to record in our db that a seeding of a certain type has occured. Make sure to register it as entity in your TypeOrmModule.forRoot call.
// file: src/seeding/entities/Seeding.entity.ts
import { Entity, PrimaryColumn, CreateDateColumn } from 'typeorm';
@Entity()
export class Seeding {
@PrimaryColumn()
public id: string;
@CreateDateColumn()
creationDate: Date;
constructor(id?: string) {
this.id = id;
}
}
An alternative seeding solution using lifecycle events:
with Nest.js, you can also implement the OnApplicationBootstrap interface (see lifecycle events) instead of going for a middleware-based solution to handle your seedings. The onApplicationBootstrap method will "called once the application has fully started and is bootstrapped". This approach, however, in contrast to a middleware-solution, will not allow you to seed your db in a multi-tenant environment where db-schemas for different tenants will be created at runtime and seeding needs to be conducted several times at runtime for different tenants after they are created.
In Nest.js, this is what B12Toaster's alternative solution using OnApplicationBootstrap could look like.
src/seeding.service.ts
import { Injectable, Logger } from '@nestjs/common';
import { EntityManager } from 'typeorm';
import { UserEntity} from 'src/entities/user.entity';
import { RoleEntity } from 'src/entities/role.entity';
import { userSeeds } from 'src/seeds/user.seeds';
import { roleSeeds } from 'src/seeds/role.seeds';
@Injectable()
export class SeedingService {
constructor(
private readonly entityManager: EntityManager,
) {}
async seed(): Promise<void> {
// Replace with your own seeds
await Promise.all([
this.entityManager.save(UserEntity, userSeeds),
this.entityManager.save(RoleEntity, roleSeeds),
]);
}
}
src/app.module.ts
import { Module, OnApplicationBootstrap } from '@nestjs/common'
import { TypeOrmModule } from '@nestjs/typeorm';
import { getConnectionOptions } from 'typeorm';
@Module({
imports: [
TypeOrmModule.forRootAsync({
useFactory: async () =>
Object.assign(await getConnectionOptions(), {
autoLoadEntities: true,
}),
}),
TypeOrmModule.forFeature([
CompanyOrmEntity,
ProductOrmEntity,
]),
],
providers: [
SeedingService,
...
],
...
})
export class AppModule implements OnApplicationBootstrap {
constructor(
private readonly seedingService: SeedingService,
) {}
async onApplicationBootstrap(): Promise<void> {
await this.seedingService.seed();
}
}
I would love to see such functionality as well (and we're not alone), but at the moment, there's no official feature for seeding.
in lack of such a built-in feature, I think the next best thing would be to create a migration script named 0-Seed (so it precedes any other migration scripts you might have) and have the seed data populated there.
@bitwit has created a snippet that may come in handy for you; it's a function that reads the data from yaml files, which you can incorporate in the seed migration script.
after some research, however, I found another interesting approach: bind an after_create event to the table, and initialize the data in the listener.
I haven't implemented this, so I'm not sure it can be done directly with TypeORM.
Unfortunately, there is no officially released solution from TypeORM (at the time this answer was being published).
But there is a nice workaround we can use:
- create another connection inside
ormconfig.jsfile and specify another folder for "migrations" - in fact our seeds - generate and run your seeds with
-c <connection name>. That's it!
Sample ormconfig.js:
module.exports = [
{
...,
migrations: [
'src/migrations/*.ts'
],
cli: {
migrationsDir: 'src/migrations',
}
},
{
name: 'seed',
...,
migrations: [
'src/seeds/*.ts'
],
cli: {
migrationsDir: 'src/seeds',
}
}
]
Sample package.json:
{
...
scripts: {
"seed:generate": "ts-node typeorm migration:generate -c seed -n ",
"seed:run": "ts-node typeorm migration:run -c seed",
"seed:revert": "ts-node typeorm migration:revert -c seed",
},
...
}