當已經利用Spark往Redis寫入數據,這部分需自行查看資料,比較簡單:
// 調用 sparkContext 實例化進行計算
val sc: SparkContext = {
val sparkConf = new SparkConf()
.setAppName(this.getClass.getSimpleName.stripSuffix("$"))
.setMaster("local[3]")
.set("spark.redis.host", "192.168.88.1xx") //redis 主機節點
.set("spark.redis.port", "6379") //端口號,不填默認為6379
.set("spark.redis.auth", "123456")
new SparkContext(sparkConf)
}
sc.setLogLevel("WARN")
// 求data
val stEndRdd: RDD[(Int, Int)] = sc.makeRDD(stEndArr)
val allPathSeq: RDD[(String, String)] = stEndRdd.map(getGraphCalcu(_)(g)) // 函數操作
// 寫入redis
sc.toRedisKV(RddData)
寫入數據后,key 和 value 均為字符串類型
這時,如果通過 Spring Boot jquery 之 ajax 直接讀取 redis 里面的數據
Object o = redisTemplate.opsForValue().get("1");
System.out.println(o);
報異常:
org.springframework.data.redis.serializer.SerializationException: Could not read JSON:
Unexpected character (',' (code 44)): Expected space separating root-level values
at [Source: (byte[])"33,1232,1231,1230,1229,1228,1227,324,1438,1439,1440"; line: 1, column: 4]; nested exception is com.fasterxml.jackson.core.JsonParseException: Unexpected character (',' (code 44)): Expected space separating root-level values
可以看到數據出現了,但是沒有讀出來,初步估計是 RedisTemplate 出現了問題。
在SpringBoot 中注解 @Configuration 的模板部分出現了問題
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
@Configuration
public class RedisConfig {
@Bean
@SuppressWarnings("all")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
template.setConnectionFactory(factory);
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
jackson2JsonRedisSerializer.setObjectMapper(om);
StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
// key采用String的序列化方式
template.setKeySerializer(stringRedisSerializer);
// hash的key也采用String的序列化方式
template.setHashKeySerializer(stringRedisSerializer);
// value序列化方式采用jackson,
template.setValueSerializer(jackson2JsonRedisSerializer);
// hash的value序列化方式采用jackson
template.setHashValueSerializer(jackson2JsonRedisSerializer);
template.afterPropertiesSet();
return template;
}
}
僅僅需要將 // value序列化方式采用jackson, 改成 stringRedisSerializer 類型就能讀出來了。
import com.fasterxml.jackson.annotation.JsonAutoDetect;
import com.fasterxml.jackson.annotation.PropertyAccessor;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
@Configuration
public class RedisConfig {
@Bean
@SuppressWarnings("all")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory factory) {
RedisTemplate<String, Object> template = new RedisTemplate<String, Object>();
template.setConnectionFactory(factory);
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
ObjectMapper om = new ObjectMapper();
om.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
om.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
jackson2JsonRedisSerializer.setObjectMapper(om);
StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
// key采用String的序列化方式
template.setKeySerializer(stringRedisSerializer);
// hash的key也采用String的序列化方式
template.setHashKeySerializer(stringRedisSerializer);
// value序列化方式采用 String
template.setValueSerializer(stringRedisSerializer);
// hash的value序列化方式采用String
template.setHashValueSerializer(stringRedisSerializer);
template.afterPropertiesSet();
return template;
}
}
關於SpringBoot 如何直接讀取Redis 里面的數據就不介紹了,網上有很多教程哪。
左側有個推薦,有用就推薦下吧?☺