What it takes to bring an AI-generated application to production

Applicability of This Document

Ready-to-Run Commands

# 対象: 自組織のアプリケーションリポジトリ（言語不問）
# 権限: リポジトリの読み取り権限
# 変更作業: なし（検索のみ）
# Production 実行: 該当なし（手元またはCIで実行）

# 1) 代表的なキー名に値が直接代入されていないかを探す
#    環境変数から読んでいる行は除外して、残ったものを目視で確認する
grep -rInE "(api[_-]?key|secret|password|passwd|token|access[_-]?key)[[:space:]]*[:=]" --exclude-dir=.git --exclude-dir=node_modules --exclude-dir=vendor . | grep -vF -e "process.env" -e "os.environ" -e "getenv" -e ".example" -e ".sample"

# 2) 秘密鍵そのものが混入していないか
grep -rIn -- "-----BEGIN" --exclude-dir=.git .

# 3) 接続文字列の形（ユーザー名とパスワードが埋め込まれたURL）を探す
grep -rInE "(postgres|mysql|mongodb|redis|amqp)://[^:@/]+:[^@/]+@" --exclude-dir=.git .

# 4) 現在のコードから消しても履歴には残る。過去のコミットも確認する
git log --oneline -S "BEGIN PRIVATE KEY" | head -n 20

# 対象: Node.js / Python のアプリケーションリポジトリ
# 権限: リポジトリの読み取り権限とパッケージレジストリへの通信
# 変更作業: なし（監査のみ）
# Production 実行: 該当なし（CIまたは開発環境で実行）

# Node.js: ロックファイルを基準に既知の脆弱性を照合する
npm audit --audit-level=high

# Python: requirements もしくはインストール済み環境を監査する
pip-audit -r requirements.txt

# コンテナで配布する場合は、ベースイメージ側のパッケージも別途監査対象になる
# （アプリの依存監査だけではOSパッケージの脆弱性は検出されない）

How to Read the Results

When to Use This

• It works locally, but once it is time to go to production you don't know where to start
• Production API keys are written directly into configuration files
• The development and production environments point at the same database
• Errors occur with no notification to anyone, and you only find out from user reports
• Deploys work, but there is no procedure for reverting to the previous version
• Unexpected cloud charges turn up at the end of the month

Possible Causes (Most Likely First)

1. 01

   Generation is aimed only at "making it work"

   The instructions given to AI are usually "implement this feature," which does not include environment separation or monitoring hooks. Anything not in the instructions is not in the output, so everything beyond the feature itself tends to be systematically missing.

2. 02

   Sample-code conventions are carried over unchanged

   Learning-material samples hardcode keys for brevity, skip error handling, and assume a single environment. Generated output can inherit these same conventions.

3. 03

   Operational requirements were never written down

   Without defined availability targets, recovery objectives, log retention periods, and cost caps, there is no way to implement them. Without requirements, you can't even detect what's missing.

4. 04

   There is no staging environment equivalent to production

   Without a staging environment, or with one configured differently from production, you cannot rehearse migrations or rollbacks. A procedure you have never tried will fail for the first time in production.

Diagnostic Steps

1. 1

   Search the repository for secrets

   Read-only

   Use the grep commands above to find hardcoded keys. Treat any key you find as something to be revoked.

2. 2

   Cross-check connection targets across environments

   Read-only

   Line up each environment's config files or environment variables and confirm that database, external API, and storage endpoints are separated per environment.

3. 3

   Audit dependency packages

   Read-only

   Run `npm audit` or `pip-audit` and review the highest-severity findings first.

4. 4

   Enumerate and check endpoints that require authentication

   Low

   Call each endpoint without an auth token to check for anything unintentionally exposed. Do this in a test environment.

5. 5

   Run a migration rollback in a test environment

   Medium

   Run apply and rollback once through to confirm no data loss occurs. Do not do this in production.

6. 6

   Restore from backup and start the app

   Medium

   Restore an existing backup into a separate environment and confirm the app starts and key operations work.

Solutions

Differences by Version and Environment

If This Doesn't Resolve It

• Count the endpoints reachable before authentication

  Confirm there are no unauthenticated endpoints besides the health check. If there are, prioritize rate limiting and input validation.

• Sample logs to check for leaked personal data

  Pull recent logs and confirm email addresses, phone numbers, and tokens are not being output.

• Check the behavior when a dependent external API goes down

  Confirm the whole app doesn't become unresponsive when an external API times out.

• Confirm certificate expiry is included in monitoring

  Having auto-renewal configured doesn't help if there's no way to notice when it fails.

• Confirm someone other than the original operator can start and stop the system

  Hand the runbook to a different person and have them execute it, to surface assumed knowledge that was never written down.

Basis and Limitations of This Document

FAQ

Questions This Document Covers

• What is a productionization checklist for an AI application
• What is missing before publishing AI-generated code to production
• What should be checked first for AI-generated app security

What the Risk Labels Mean

• Read-onlyDoes not change data or settings.
• LowImpact is limited, but permissions and load should be checked.
• MediumMay affect performance, locking, or cost.
• HighMay cause an outage, data loss, or require recovery work.
• Expert Review RequiredRequires a separate review before applying to production.

GIIP's Scope of Support

Your organization can carry out all of the items above without outsourcing them. The judgment call is which items to satisfy before this release and which to defer. At GIIP, when bringing an AI-generated application to production, we satisfy environment separation, secrets externalization, and a measured rollback procedure, plus a monitoring integration, before publishing. If you lack the information to prioritize, we can also help by reviewing your current setup and identifying the gaps.

Related Services

• Dev / Stg / Prod の分離を設計する
• AIアプリの本番リリース支援
• AIプロダクション運用