我正在編寫一些 Go 軟件，負(fù)責(zé)下載和解析大量 JSON 文件并將解析后的數(shù)據(jù)寫入 sqlite 數(shù)據(jù)庫。我目前的設(shè)計(jì)有 10 個(gè) go 例程，同時(shí)下載/解析這些 JSON 并將它們傳送到另一個(gè) go 例程，該例程的唯一工作是監(jiān)聽特定頻道并將頻道內(nèi)容寫入數(shù)據(jù)庫。系統(tǒng)會(huì)在所有寫入完成后執(zhí)行一些額外的讀取操作，這會(huì)導(dǎo)致查詢返回錯(cuò)誤結(jié)果的問題，因?yàn)椴⒎撬袛?shù)據(jù)都已寫入表。因?yàn)槲姨崛〉?JSON 數(shù)據(jù)是動(dòng)態(tài)的，所以我沒有簡單的方法知道所有數(shù)據(jù)何時(shí)都已寫入。我考慮了兩種解決方案的可能性，盡管我對這兩種解決方案都不太滿意：在頻道上收聽并等待它為空。這原則上應(yīng)該有效，但是，它不能確保數(shù)據(jù)已被寫入，它只確保它已在通道上接收到。同步對數(shù)據(jù)庫的訪問。這在原則上應(yīng)該再次起作用，但是，我仍然需要將查詢操作放在所有寫操作之后。我是否應(yīng)該考慮其他任何設(shè)計(jì)決策來糾正這個(gè)問題？作為參考，我用來提取這些數(shù)據(jù)的庫是 go-colly 和 go-sqlite3。感謝所有的幫助！
1 回答
阿晨1998
TA貢獻(xiàn)2037條經(jīng)驗(yàn) 獲得超6個(gè)贊
你可以使用一個(gè)sync.WaitGroup
例如
package main
import "sync"
func main() {
// Some sort of job queue for your workers to process. This job queue should be closed by the process
// that populates it with items. Once the job channel is closed, any for loops ranging over the channel
// will read items until there are no more items, and then break.
jobChan := make(chan JobInfo)
// Populate the job queue here...
// ...
close(jobChan)
// We now have a full queue of jobs that can't accept new jobs because the channel is closed.
// Number of concurrent workers.
workerCount := 10
// Initialize the WaitGroup.
wg := sync.WaitGroup{}
wg.Add(workerCount)
// Create the worker goroutines.
for i := 0; i < workerCount; i++ {
go func() {
// When the jobChan is closed, and no more jobs are available on the queue, the for loop
// will exit, causing wg.Done() to be called, and the anonymous function to exit.
for job := range jobChan {
// Process job.
}
wg.Done()
}()
}
// Wait for all workers to call wg.Done()
wg.Wait()
// Whatever you want to do after all queue items have been processed goes here.
// ...
}
- 1 回答
- 0 關(guān)注
- 215 瀏覽
添加回答
舉報(bào)
0/150
提交
取消