dagster field under +meta configuration. The following are equivalent:Before:
version: 2 models: - name: users config: dagster_freshness_policy: maximum_lag_minutes: 60 cron_schedule: "0 9 * * *" dagster_auto_materialize_policy: type: "lazy"
After:
version: 2 models: - name: users config: meta: dagster: freshness_policy: maximum_lag_minutes: 60 cron_schedule: "0 9 * * *" auto_materialize_policy: type: "lazy"
Added support for Pythonic Config classes to the @configured API, which makes reusing op and asset definitions easier:
class GreetingConfig(Config): message: str @op def greeting_op(config: GreetingConfig): print(config.message) class HelloConfig(Config): name: str @configured(greeting_op) def hello_op(config: HelloConfig): return GreetingConfig(message=f"Hello, {config.name}!")
Added AssetExecutionContext to replace OpExecutionContext as the context object passed in to @asset functions.
TimeWindowPartitionMapping now contains an allow_nonexistent_upstream_partitions argument that, when set to True, allows a downstream partition subset to have nonexistent upstream parents.
Unpinned the alembic dependency in the dagster package.
[ui] A new “Assets” tab is available from the Overview page.
[ui] The Backfills table now includes links to the assets that were targeted by the backfill.
croniter==1.4.0. Users of earlier versions of Dagster can pin croniter<1.4.@run_failure_sensor.py.typed was missing in the dagster-graphql package. Thanks @Tanguy-LeFloch!AutoMaterializePolicys will now be cleared after 1 week.@dbt_assets:profile and target can now be customized on the DbtCli resource.partial_parse.msgpack is detected in the target directory of your dbt project, it is now copied into the target directories created by DbtCli to take advantage of partial parsing.@dbt_assets can now be customized by overriding DbtManifest.node_info_to_metadata.AssetMaterializations.serverK8sConfig.containerConfig.name did not actually change the container name.1.3.8 release where the Dagster Cloud agent would sometimes fail to start up with an import error.define_asset_job now accepts a hooks argument.sqlalchemy==2.xadditionalInstanceConfig key that allows you to supply additional configuration to the Dagster instance.EcsRunLauncher now uses a different task definition family for each job, instead of registering a new task definition revision each time a different job is launched.EcsRunLauncher now includes a run_ecs_tags config key that lets you configure tags on the launched ECS task for each run.SkipReason, the SkipReason would be ignored. This has been fixed.RuntimeError: dictionary changed size during iteration exception while code servers were being reloaded.dagster code-server start CLI, instead of only reloading your code when you initiate a reload from the Dagster UI.metadata parameter to define_asset_job (Thanks Elliot2718!)poll_sync_run to handle the “preparing” status from the Census API (Thanks ldnicolasmay!)@observable_source_asset-decorated functions can now return a DataVersionsByPartition to record versions for partitions.@dbt_assetsDbtCliTask's created by invoking DbtCli.cli(...) now have a method .is_successful(), which returns a boolean representing whether the underlying CLI process executed the dbt command successfully.@dbt_assets can now be customized by overriding DbtManifest.node_info_to_description.@dbt_assets.server_ecs_tags and run_ecs_tags that apply to each service or task created by the agent. See the docs for more information.instance.get_run_partition_data in Dagster Cloud..env file in the working directory when running dagster dev can now be used for Dagster system variables like DAGSTER_HOME or environment variables referenced in your dagster.yaml file using an env: key. Previously, setting a .env file only worked for environment variables referenced in your Dagster code.submit_job_execution can now take in a RunConfig object. Previously, it could only take a Python dictionary with the run configuration.jobs argument to Definitions./instance/runs path instead of the new /runs.create_databricks_run_now_op, thanks @srggrs!dagster code-server start command that can be used to launch a code server, much like dagster api grpc. Unlike dagster api grpc, however, dagster code-server start runs the code in a subprocess, so it can reload code from the Dagster UI without needing to restart the command. This can be useful for jobs that load code from some external source and may want to reload job definitions without restarting the process.sensors.num_submit_workers key to dagster.yaml that can be used to decrease latency when a sensor emits multiple run requests within a single tick. See the docs for more information.k8s_job_executor can now be used to launch each step of a job in its own Kubernetes, pod, even if the Dagster deployment is not using the K8sRunLauncher to launch each run in its own Kubernetes pod.io_manager_def param on an asset.maxResumeRunAttempts to null in the helm chart would cause it to be set to a default value of 3 instead of disabling run retries.k8s_job_executor would sometimes fail with a 409 Conflict error after retrying the creation of a Kubernetes pod for a step, due to the job having already been created during a previous attempt despite raising an error.op_name was passed to load_assets_from_dbt_manifest, and a select parameter was specified, a suffix would be appended to the desired op name.dagit would lead to JavaScript bundle loading errors.DbtCli, in dagster_dbt.cli. This new resource only support dbt-core>=1.4.0.@dbt_assets in dagster_dbt.asset_decorator that allows you to specify a compute function for a selected set of dbt assets that loaded as an AssetsDefinition.