Add new source: jsmon for subdomain discovery #1617
Conversation
WalkthroughThe changes add a new subdomain enumeration source named Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant Runner
participant jsmon.Source
participant JSMonAPI
User->>Runner: Start subdomain enumeration for domain
Runner->>jsmon.Source: Run(ctx, domain, session)
jsmon.Source->>jsmon.Source: Verify API keys
alt No API keys
jsmon.Source-->>Runner: Mark source as skipped, return
else API keys present
jsmon.Source->>JSMonAPI: POST /api/v2/subfinderScan with domain and auth headers
JSMonAPI-->>jsmon.Source: Return JSON response (subdomains or error)
alt Success
jsmon.Source->>Runner: Stream subdomain results asynchronously
else Error or invalid response
jsmon.Source->>Runner: Send error result
end
end
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
✅ Files skipped from review due to trivial changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
v2/pkg/subscraping/sources/jsmon/jsmon.go (1)
15-19: Remove unused struct definition.The
subdomainsResponsestruct is defined but never used in the code. This appears to be leftover code that should be removed to avoid confusion.-type subdomainsResponse struct { - Subdomains []string `json:"subdomains"` - Status string `json:"status"` - Message string `json:"message"` -}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
v2/pkg/passive/sources.go(2 hunks)v2/pkg/runner/options.go(1 hunks)v2/pkg/subscraping/sources/jsmon/jsmon.go(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
v2/pkg/passive/sources.go (1)
Learnt from: x-stp
PR: projectdiscovery/subfinder#0
File: :0-0
Timestamp: 2025-06-24T16:03:30.467Z
Learning: When fixing HTTP response handling bugs in subfinder sources, the correct pattern is to use `defer session.DiscardHTTPResponse(resp)` after successful requests to ensure the response body remains open for reading, and call `session.DiscardHTTPResponse(resp)` immediately in error cases.
🧬 Code Graph Analysis (2)
v2/pkg/passive/sources.go (2)
v2/pkg/subscraping/sources/jsmon/jsmon.go (1)
Source(21-27)v2/pkg/subscraping/types.go (1)
Source(37-62)
v2/pkg/subscraping/sources/jsmon/jsmon.go (1)
v2/pkg/subscraping/types.go (2)
Session(71-78)Statistics(29-34)
🔇 Additional comments (6)
v2/pkg/subscraping/sources/jsmon/jsmon.go (3)
61-71: LGTM: URL construction and request preparation.The URL construction and request body preparation logic is well-implemented. The use of query parameters for workspace ID and proper JSON formatting for the request body follows good practices.
122-149: LGTM: Interface method implementations.All required interface methods are properly implemented with appropriate return values. The source correctly indicates it needs API keys, supports recursive enumeration, and is not enabled by default.
72-79: JSMon HTTP response cleanup is correctly implemented
Thejsmonsource callssession.DiscardHTTPResponse(resp)immediately on every error path and once on the success path before processing results, matching the pattern used throughout other sources. No changes needed.v2/pkg/runner/options.go (1)
258-258: LGTM: Netlas rate limit removal.The netlas rate limit has been properly commented out with a clear explanation. This change aligns with the removal of the netlas source from the passive sources list.
v2/pkg/passive/sources.go (2)
37-37: LGTM: Jsmon import addition.The import for the jsmon source is correctly placed in alphabetical order within the existing imports.
69-69: LGTM: Jsmon source registration.The jsmon source is properly registered in the AllSources array following the established pattern. The placement maintains the existing structure of the array.
| var subdomains []string | ||
| err = jsoniter.NewDecoder(resp.Body).Decode(&subdomains) | ||
| if err != nil { | ||
| // Read response body for debugging | ||
| body, _ := io.ReadAll(resp.Body) | ||
| fmt.Printf("[DEBUG] Response body: %s\n", string(body)) | ||
| fmt.Printf("[DEBUG] JSON decode error: %v\n", err) | ||
| results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err} | ||
| s.errors++ | ||
| session.DiscardHTTPResponse(resp) | ||
| return | ||
| } |
There was a problem hiding this comment.
Remove debug print statements and improve error handling.
The code contains debug print statements that should not be present in production code. Additionally, the JSON parsing logic expects a direct array of subdomains, which contradicts the unused subdomainsResponse struct.
Apply this diff to remove debug statements and improve error handling:
- // Parse the response as a direct array of subdomains
var subdomains []string
err = jsoniter.NewDecoder(resp.Body).Decode(&subdomains)
if err != nil {
- // Read response body for debugging
- body, _ := io.ReadAll(resp.Body)
- fmt.Printf("[DEBUG] Response body: %s\n", string(body))
- fmt.Printf("[DEBUG] JSON decode error: %v\n", err)
results <- subscraping.Result{Source: s.Name(), Type: subscraping.Error, Error: err}
s.errors++
session.DiscardHTTPResponse(resp)
return
}Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In v2/pkg/subscraping/sources/jsmon/jsmon.go around lines 94 to 105, remove the
debug print statements that output the response body and JSON decode error, as
these should not be in production code. Instead, improve error handling by
properly using the existing subdomainsResponse struct to parse the JSON
response, ensuring the JSON decoding matches the expected structure. Adjust the
decoding logic to decode into the subdomainsResponse struct and extract
subdomains from it, then handle errors by sending them through the results
channel and discarding the HTTP response as before.
|
Please take a look at the following
|
This PR adds a new passive source named 'jsmon' for discovering subdomains.
Changes made:
Tested and verified output using the 'subfinder' tool. This source contributes to better subdomain coverage using jsmon data.
Looking forward to feedback or suggestions from the maintainers.
Summary by CodeRabbit