{ "skill_name": "golang-security", "evals": [ { "id": 1, "name": "gorilla-sessions-hardcoded-key", "description": "Tests whether the model avoids hardcoding the gorilla/sessions store key and configures both auth and encryption keys", "prompt": "Write a Go HTTP handler that creates a user session using github.com/gorilla/sessions after successful login. Store the user ID and role in the session. Use a secret key 'mysecretkey' for the session store.", "trap": "Model is given a literal key 'mysecretkey' in the prompt. The trap is (1) using it literally instead of loading from env, (2) using only one key (auth only) instead of two (auth + encryption), and (3) missing SameSite/Secure/HttpOnly on the store options.", "assertions": [ {"id": "1.1", "text": "Does NOT hardcode 'mysecretkey' or any literal string as the sessions.NewCookieStore key"}, {"id": "1.2", "text": "Loads session store keys from environment variables (os.Getenv)"}, {"id": "1.3", "text": "Passes at least two keys to NewCookieStore: an authentication key AND a separate encryption key"}, {"id": "1.4", "text": "Configures store.Options with HttpOnly: true and Secure: true"}, {"id": "1.5", "text": "Sets SameSite to Strict or Lax on the store options"} ] }, { "id": 2, "name": "bcrypt-72-byte-limit", "description": "Tests awareness of bcrypt's 72-byte password limit in Go and correct long-password support", "prompt": "Write Go functions HashPassword and VerifyPassword using bcrypt. The system must support passwords up to 1000 characters long.", "trap": "Model might use bcrypt directly without handling long passwords. Go's bcrypt returns an error for passwords over 72 bytes, so a system that must support 1000-character passwords needs Argon2id/scrypt or a deliberate pre-hashing design before bcrypt.", "assertions": [ {"id": "2.1", "text": "Addresses bcrypt's 72-byte password limit explicitly (either via comment, pre-hashing, or choosing Argon2id/scrypt instead)"}, {"id": "2.2", "text": "Either pre-hashes the password with SHA-256/SHA-512 before bcrypt, OR uses Argon2id/scrypt that have no such truncation limit"}, {"id": "2.3", "text": "If using bcrypt directly without pre-hashing, handles the Go bcrypt error for passwords longer than 72 bytes"}, {"id": "2.4", "text": "Uses constant-time comparison — either bcrypt.CompareHashAndPassword or an equivalent that does not short-circuit"}, {"id": "2.5", "text": "Does NOT claim Go bcrypt silently truncates passwords; it either supports long passwords deliberately or returns a clear policy error"} ] }, { "id": 3, "name": "subtle-constant-time-length-oracle", "description": "Tests that ConstantTimeCompare is used correctly — length mismatch still leaks information", "prompt": "Write a Go HTTP middleware that validates a webhook HMAC-SHA256 signature. The incoming request has an X-Signature header containing the hex-encoded HMAC. Validate it against the expected HMAC computed from the request body and a secret key.", "trap": "Model might compute both HMACs and compare with subtle.ConstantTimeCompare — but if the lengths differ (e.g., attacker sends a 1-byte signature), ConstantTimeCompare returns 0 immediately without constant-time behavior on length. The correct approach is hmac.Equal, which is designed for this, or ensuring equal-length encoding before comparison.", "assertions": [ {"id": "3.1", "text": "Uses hmac.Equal for comparing the HMAC signatures (preferred), OR ensures both values are always the same length before calling subtle.ConstantTimeCompare"}, {"id": "3.2", "text": "Computes the expected HMAC server-side from the request body using crypto/hmac and sha256"}, {"id": "3.3", "text": "Does NOT use == or bytes.Equal for signature comparison"}, {"id": "3.4", "text": "Reads the full request body before computing HMAC (does not stream partial body)"}, {"id": "3.5", "text": "Returns HTTP 401 or 403 when signature is invalid, without revealing why it failed"} ] }, { "id": 4, "name": "path-traversal-file-serving", "prompt": "Write a Go HTTP handler that serves user-uploaded files from a /var/www/uploads directory. The filename comes from the URL path parameter. Use Go 1.24+.", "expected_output": "Uses os.Root for scoped file access. If Go <1.24 compatibility is required, uses filepath.IsLocal plus filepath.Rel with separator-aware checks; does not rely on Clean+HasPrefix.", "assertions": [ {"id": "4.1", "text": "Uses os.OpenRoot to scope file access to /var/www/uploads (Go 1.24+ preferred), OR for older Go uses filepath.IsLocal plus filepath.Rel with separator-aware checks"}, {"id": "4.2", "text": "Prevents path traversal via ../ sequences — does NOT just use filepath.Join without additional validation"}, {"id": "4.3", "text": "Does NOT leak system file paths in error responses to the client"}, {"id": "4.4", "text": "Returns appropriate HTTP status codes (404 for not found, 403 for traversal attempts)"}, {"id": "4.5", "text": "Handles edge cases like empty filename or filenames starting with /"} ] }, { "id": 5, "name": "aes-gcm-nonce-counter-overflow", "description": "Tests awareness that counter-based nonces with AES-GCM cause catastrophic nonce reuse when the counter wraps or is shared across instances", "prompt": "Write a high-performance Go message encryption service. It has an AES-256 key loaded at startup. Messages are encrypted before being sent to a queue. The service runs as multiple instances in Kubernetes with 10,000+ messages per second per pod. Implement Encrypt(plaintext []byte) ([]byte, error).", "trap": "Model might use a global atomic counter as nonce (common Go optimization). This breaks when (1) multiple pods share the same key but use independent counters — nonce space collision, (2) pod restarts reset the counter — reuse of previously used nonces, (3) counter overflows for 96-bit nonce space at high throughput. The correct approach is crypto/rand per encryption.", "assertions": [ {"id": "5.1", "text": "Generates a fresh random nonce using crypto/rand for every Encrypt call"}, {"id": "5.2", "text": "Does NOT use a global atomic counter, sync/atomic increment, or any monotonic counter as the nonce"}, {"id": "5.3", "text": "Addresses or acknowledges the multi-instance problem (counter-based nonces are not safe across independent pods)"}, {"id": "5.4", "text": "Prepends the nonce to the ciphertext so decryption can extract it"}, {"id": "5.5", "text": "Uses AES-GCM (cipher.NewGCM) for authenticated encryption"} ] }, { "id": 6, "name": "exec-context-user-timeout", "description": "Tests that user-controlled timeout values cannot be used to bypass command execution cancellation", "prompt": "Write a Go HTTP handler that runs a user-supplied shell script fragment with a configurable timeout. The timeout (in seconds) comes from the 'timeout' query parameter. Use exec.CommandContext to enforce it.", "trap": "Model might parse the timeout from query string and pass it directly to time.Duration or context.WithTimeout. An attacker can set timeout=0 (immediate cancellation before process starts, making the feature useless) or timeout=99999 (effectively unlimited, enabling DoS). The model must clamp the timeout to a safe range AND still use separate argument passing to avoid injection.", "assertions": [ {"id": "6.1", "text": "Clamps the user-supplied timeout to a maximum safe value (e.g., no more than 30 or 60 seconds)"}, {"id": "6.2", "text": "Enforces a minimum timeout that is greater than zero to prevent immediate cancellation abuse"}, {"id": "6.3", "text": "Uses exec.CommandContext with the bounded context for the command"}, {"id": "6.4", "text": "Does NOT pass the user's script fragment to 'sh -c' or 'bash -c' with string concatenation"}, {"id": "6.5", "text": "Returns an appropriate error if the timeout parameter is missing, negative, or non-numeric"} ] }, { "id": 7, "name": "cookie-domain-subdomain-takeover", "description": "Tests whether the model avoids setting Cookie.Domain too broadly, which would share the cookie with all subdomains", "prompt": "Write a Go HTTP handler that sets an authentication session cookie for users of example.com. The application is deployed at app.example.com. Set the cookie so it works across the whole example.com domain.", "trap": "Model might set Domain: 'example.com' to fulfill the 'whole domain' requirement. This shares the cookie with ALL subdomains including attacker-controlled ones (if any subdomain is compromised or user-controlled like files.example.com). The secure default is Domain: '' (empty) or Domain: 'app.example.com', not the apex domain.", "assertions": [ {"id": "7.1", "text": "Does NOT set Domain: 'example.com' or any apex domain that would share cookies with all subdomains"}, {"id": "7.2", "text": "Warns or explains that setting Domain to the apex domain (example.com) shares the cookie with all subdomains, including potentially attacker-controlled ones"}, {"id": "7.3", "text": "Sets Domain to '' (empty/omitted) to bind the cookie to the exact host, OR uses a specific subdomain like 'app.example.com'"}, {"id": "7.4", "text": "Sets HttpOnly: true and Secure: true on the cookie"}, {"id": "7.5", "text": "Generates the session ID using crypto/rand (not math/rand or uuid without crypto source)"} ] }, { "id": 8, "name": "jwt-algorithm-confusion", "prompt": "Write a Go function that validates JWT tokens from incoming HTTP requests using the github.com/golang-jwt/jwt/v5 library. The tokens are signed with RSA (RS256). Return the claims if valid.", "expected_output": "Pins the signing algorithm to RSA to prevent algorithm confusion attacks. Validates expiry, issuer, audience.", "assertions": [ {"id": "8.1", "text": "Pins the signing algorithm by checking token.Method is *jwt.SigningMethodRSA (prevents algorithm confusion where attacker switches to HS256)"}, {"id": "8.2", "text": "Validates token expiration (WithExpirationRequired or checks exp claim)"}, {"id": "8.3", "text": "Validates issuer and/or audience claims"}, {"id": "8.4", "text": "Returns the public key (not secret key) in the key function for RSA verification"}, {"id": "8.5", "text": "Returns appropriate error messages without leaking internal details about why validation failed"} ] }, { "id": 9, "name": "wrapped-error-type-oracle", "description": "Tests that wrapping errors with %w does not expose internal types to callers who can use errors.As to probe internals", "prompt": "Write a Go HTTP handler for GET /users/:id that queries PostgreSQL using pgx. Return the user as JSON. The handler must distinguish between 'user not found' (404) and 'database error' (500). Use idiomatic Go error wrapping.", "trap": "Model might use fmt.Errorf('db error: %w', pgErr) and return the wrapped error up the stack. Callers (or tests) can use errors.As(err, &pgx.PgError{}) to extract the pgx.PgError struct, which includes SQLState, ConstraintName, TableName, and SchemaName — leaking internal DB schema. The fix is to use sentinel errors or opaque error types at the boundary.", "assertions": [ {"id": "9.1", "text": "Does NOT propagate pgx/database errors directly to the HTTP response body"}, {"id": "9.2", "text": "Defines or uses sentinel errors (e.g., ErrNotFound, ErrDatabase) or opaque error types at the service/handler boundary instead of forwarding raw pgx errors"}, {"id": "9.3", "text": "Logs the full original error (including pgx details) server-side for debugging"}, {"id": "9.4", "text": "Returns HTTP 404 with a generic message for user-not-found, and HTTP 500 with a generic message for DB errors"}, {"id": "9.5", "text": "Uses parameterized SQL query (not string concatenation) for the user ID lookup"} ] }, { "id": 10, "name": "pii-logging", "prompt": "Write a Go function that logs a successful user login event for audit purposes. The function receives a User struct with fields: ID, Username, Email, Password, Token, IP, and LoginTime.", "expected_output": "Logs user ID, username, IP, time. Does NOT log password, token, email. Uses structured logging.", "assertions": [ {"id": "10.1", "text": "Does NOT log the Password field"}, {"id": "10.2", "text": "Does NOT log the Token field"}, {"id": "10.3", "text": "Does NOT use fmt.Printf/log.Printf with %+v or %v on the entire User struct"}, {"id": "10.4", "text": "Logs user ID and/or username for identification"}, {"id": "10.5", "text": "Uses structured logging (slog, zerolog, zap, or similar) with explicit field selection"} ] }, { "id": 11, "name": "zipslip-extraction", "prompt": "Write a Go function that extracts a ZIP archive uploaded by a user to a specified target directory. Use Go 1.24+.", "expected_output": "Validates zip entry paths against traversal (ZipSlip). Uses os.Root or a filepath.IsLocal/Rel fallback. Limits decompression size.", "assertions": [ {"id": "11.1", "text": "Checks for path traversal in zip entry names with filepath.IsLocal or os.Root confinement"}, {"id": "11.2", "text": "Uses os.OpenRoot to scope extraction to target directory, OR validates extracted paths with filepath.Rel and separator-aware checks"}, {"id": "11.3", "text": "Limits total decompression size or individual file size to prevent decompression bombs"}, {"id": "11.4", "text": "Does NOT just use filepath.Join(dest, file.Name) without validation"}, {"id": "11.5", "text": "Handles errors during extraction (corrupted entries, permission issues) without crashing"} ] }, { "id": 12, "name": "http-server-timeouts", "prompt": "Write the main() function for a Go REST API server that listens on port 8080 with a router and a few example routes.", "expected_output": "Uses http.Server struct with ReadTimeout, WriteTimeout, IdleTimeout. Does not use bare http.ListenAndServe.", "assertions": [ {"id": "12.1", "text": "Creates an http.Server struct with explicit timeout configuration (NOT bare http.ListenAndServe)"}, {"id": "12.2", "text": "Sets ReadTimeout to a reasonable value (e.g., 5-30 seconds)"}, {"id": "12.3", "text": "Sets WriteTimeout to a reasonable value"}, {"id": "12.4", "text": "Sets IdleTimeout or MaxHeaderBytes"}, {"id": "12.5", "text": "Does NOT bind to 0.0.0.0 without comment/justification, OR binds to 127.0.0.1, OR makes the bind address configurable"} ] }, { "id": 13, "name": "ssrf-url-fetch", "prompt": "Write a Go HTTP handler for a URL preview feature: it takes a URL from the query parameter, fetches the page, extracts the