我在從 MongoDB 2 Azure blob 存儲(chǔ)遷移文件時(shí)遇到問題。下一個(gè)方法獲取一個(gè) GridFSFile 對(duì)象（代表 MongoDB GridFSFileStorage 中的一個(gè)文件），然后調(diào)用 uploadMemoryStream 方法進(jìn)行上傳。值得一提的是，gridFSFile 在 findById 之后確實(shí)有內(nèi)容，并且 length dows 也有內(nèi)容，并且該位置最初在 0。gridFSFile.Open 方法創(chuàng)建一個(gè) Stream 對(duì)象，然后我將其作為參數(shù)傳遞給上傳。private static void iterateOverVersionCollection(Version version, Asset asset){ try { string _gridFSId = version.GridFSId; GridFSFile gridFSFile = gridFSFileStorage.FindById(_gridFSId); if (gridFSFile == null) return; string size = version.Name.ToLower(); asset.Size = size; CloudBlockBlob blockBlob = GetBlockBlobReference(version, gridFSFile, asset); uploadMemoryStream(blockBlob, gridFSFile, asset); asset.UploadedOK = true; } catch (StorageException ex) { asset.UploadedOK = false; logException(ex, asset); }}private static void uploadMemoryStream(CloudBlockBlob blockBlob, GridFSFile gridFSFile, Asset asset){ Stream st = gridFSFile.Open(); blockBlob.UploadFromStream(st);}UploadFromStream 需要永遠(yuǎn)并且永遠(yuǎn)不會(huì)上傳，而且要提到的一件事是，無論我如何使用 gridFSFile，如果我嘗試使用 Stream.copyTo c# 方法用它創(chuàng)建一個(gè) MemoryStream，它也需要永遠(yuǎn)并且永無止境所以應(yīng)用程序卡在 blockBlob.UploadFromStream(st);除了將 gridFSFile.Open 傳遞給 UploadFromMemoryStream 我還嘗試了下一段代碼：using (var stream = new MemoryStream()){ byte[] buffer = new byte[2048]; // read in chunks of 2KB int bytesRead; while((bytesRead = st.Read(buffer, 0, buffer.Length)) > 0) { stream.Write(buffer, 0, bytesRead); } byte[] result = stream.ToArray();}但同樣，程序卡在 st.Read 行。任何幫助都感激不盡。
1 回答
拉丁的傳說
TA貢獻(xiàn)1789條經(jīng)驗(yàn) 獲得超8個(gè)贊
請(qǐng)注意，由于 UploadFromFileAsync() 或 UploadFromStream 對(duì)于巨大的 blob 不是可靠且高效的操作，我建議您考慮以下替代方案：
如果你可以接受命令行工具，你可以嘗試 AzCopy，它能夠以高性能傳輸 Azure 存儲(chǔ)數(shù)據(jù)，并且可以暫停和恢復(fù)傳輸。
如果您想以編程方式控制傳輸作業(yè)，請(qǐng)使用Azure Storage Data Movement Library，它是 AzCopy 的核心。相同的示例代碼
string storageConnectionString = "myStorageConnectionString";
CloudStorageAccount account = CloudStorageAccount.Parse(storageConnectionString);
CloudBlobClient blobClient = account.CreateCloudBlobClient();
CloudBlobContainer blobContainer = blobClient.GetContainerReference("mycontainer");
blobContainer.CreateIfNotExistsAsync().Wait();
string sourcePath = @"C:\Tom\TestLargeFile.zip";
CloudBlockBlob destBlob = blobContainer.GetBlockBlobReference("LargeFile.zip");
// Setup the number of the concurrent operations
TransferManager.Configurations.ParallelOperations = 64;
// Setup the transfer context and track the upoload progress
var context = new SingleTransferContext
{
ProgressHandler =
new Progress<TransferStatus>(
progress => { Console.WriteLine("Bytes uploaded: {0}", progress.BytesTransferred); })
};
// Upload a local blob
TransferManager.UploadAsync(sourcePath, destBlob, null, context, CancellationToken.None).Wait();
Console.WriteLine("Upload finished !");
Console.ReadKey();
如果您仍在尋找從流中以編程方式上傳文件，我建議您使用以下代碼分塊上傳
var container = _client.GetContainerReference("test");
container.CreateIfNotExists();
var blob = container.GetBlockBlobReference(file.FileName);
var blockDataList = new Dictionary<string, byte[]>();
using (var stream = file.InputStream)
{
var blockSizeInKB = 1024;
var offset = 0;
var index = 0;
while (offset < stream.Length)
{
var readLength = Math.Min(1024 * blockSizeInKB, (int)stream.Length - offset);
var blockData = new byte[readLength];
offset += stream.Read(blockData, 0, readLength);
blockDataList.Add(Convert.ToBase64String(BitConverter.GetBytes(index)), blockData);
index++;
}
}
Parallel.ForEach(blockDataList, (bi) =>
{
blob.PutBlock(bi.Key, new MemoryStream(bi.Value), null);
});
blob.PutBlockList(blockDataList.Select(b => b.Key).ToArray());
另一方面，如果您的系統(tǒng)中有可用的文件并想使用 Uploadfile 方法，我們也可以靈活地使用此方法以塊的形式上傳文件數(shù)據(jù)
TimeSpan backOffPeriod = TimeSpan.FromSeconds(2);
int retryCount = 1;
BlobRequestOptions bro = new BlobRequestOptions()
{
SingleBlobUploadThresholdInBytes = 1024 * 1024, //1MB, the minimum
ParallelOperationThreadCount = 1,
RetryPolicy = new ExponentialRetry(backOffPeriod, retryCount),
};
CloudStorageAccount cloudStorageAccount = CloudStorageAccount.Parse(ConnectionString);
CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
cloudBlobClient.DefaultRequestOptions = bro;
cloudBlobContainer = cloudBlobClient.GetContainerReference(ContainerName);
CloudBlockBlob blob = cloudBlobContainer.GetBlockBlobReference(Path.GetFileName(fileName));
blob.StreamWriteSizeInBytes = 256 * 1024; //256 k
blob.UploadFromFile(fileName, FileMode.Open);
詳細(xì)解釋請(qǐng)瀏覽
https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/
希望能幫助到你。
- 1 回答
- 0 關(guān)注
- 169 瀏覽
添加回答
舉報(bào)
0/150
提交
取消