Jsoup下载抖音文件限制1M的问题解决-白红宇

1.最近有个朋友找我需要下载抖音文件,我研究了下,首先想到用的是用jsoup这个组件进行下载。

2.项目中一般都是用maven开发的 ,首先要引入jsoup的gav，还用到了commons-io组件。


    	    
     
      org.jsoup
     	    
     
      jsoup
     	    
     
      1.11.2
     	
    	
    		
     
      commons-io
     		
     
      commons-io
     		
     
      2.6
     	
    复制代码

3.代码干货：

package com.vhd; import java.io.BufferedInputStream;import java.io.File;import java.io.IOException; import org.apache.commons.io.FileUtils;import org.apache.commons.lang3.StringUtils;import org.jsoup.Connection.Response;import org.jsoup.Jsoup;import org.slf4j.Logger;import org.slf4j.LoggerFactory; public class DouYinUtil1 { 	private final static Logger log = LoggerFactory			.getLogger(DouYinUtil.class);	 	public static void getDouYinUrl(String douyinUrl,String fileName){		try {			String content = Jsoup.connect(douyinUrl).ignoreContentType(true).timeout(10000).execute().body();			String downUrl = StringUtils.substringBetween(content, "playAddr: \"", "\",");			if(StringUtils.isEmpty(downUrl)){				return;			}			Response document = Jsoup.connect(downUrl).timeout(10000).execute();			BufferedInputStream stream = document.bodyStream();			File file = new File(fileName);			FileUtils.copyInputStreamToFile(stream, file);		} catch (IOException e) {			e.printStackTrace();			log.error("douyinUrl:{},error:{}",douyinUrl,e);		}	}	}复制代码

比如爬去这个链接下的视频,并把他下载到本地：

结果下载之后只有1M的大小.就研究了下jsouip这个组件,测试只有下载1M的文件：

String url = "https://aweme.snssdk.com/aweme/v1/playwm/?video_id=v0300fd30000bi2c7ipdja5i7hebu8d0&line=0";	System.out.println(Jsoup.connect(url).request().maxBodySize());复制代码

然后查看API文档：

Connection maxBodySize(int bytes) Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated.

可以设置下载文件的大下.

然后在上面代码基础增加了.maxBodySize(3000000)这个方法就可以限制1M大小的问题了。

package com.vhd; import java.io.BufferedInputStream;import java.io.File;import java.io.IOException; import org.apache.commons.io.FileUtils;import org.apache.commons.lang3.StringUtils;import org.jsoup.Connection.Response;import org.jsoup.Jsoup;import org.slf4j.Logger;import org.slf4j.LoggerFactory; public class DouYinUtil { 	private final static Logger log = LoggerFactory			.getLogger(DouYinUtil.class);	 	public static void getDouYinUrl(String douyinUrl,String fileName){		try {			String content = Jsoup.connect(douyinUrl).ignoreContentType(true).timeout(10000).execute().body();			String downUrl = StringUtils.substringBetween(content, "playAddr: \"", "\",");			if(StringUtils.isEmpty(downUrl)){				return;			}			Response document = Jsoup.connect(downUrl).maxBodySize(30000000).timeout(10000).execute();			BufferedInputStream stream = document.bodyStream();			File file = new File(fileName);			FileUtils.copyInputStreamToFile(stream, file);            //此方法建议可以使用下面copy大文件的方法性能会更好些            //IOUtils.copyLarge(stream, new FileOutputStream("D:\\123.mp4"));		} catch (IOException e) {			e.printStackTrace();			log.error("douyinUrl:{},error:{}",douyinUrl,e);		}	}		public static void main(String[] args) {		String url = "https://aweme.snssdk.com/aweme/v1/playwm/?video_id=v0300fd30000bi2c7ipdja5i7hebu8d0&line=0";		System.out.println(Jsoup.connect(url).request().maxBodySize());	}}复制代码

以上帮朋友解决了下载抖音大文件的问题,也可以做成批量下载抖音,有兴趣可以尝试一下！

欢迎关注我的公众号，来一起学习：